[jira] [Commented] (HBASE-10396) The constructor of HBaseAdmin may close the shared HConnection
[
https://issues.apache.org/jira/browse/HBASE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882231#comment-13882231
]
cuijianwei commented on HBASE-10396:
[~stack] Thanks for your concern. I make a patch for 0.94 to use
HConnectionManager#deleteConnection(Configuration) so that will not delete the
shared HConnection if other threads are using it. BTW, There are two
constructors of HBaseAdmin:
{code}
public HBaseAdmin(Configuration c) throws MasterNotRunningException,
ZooKeeperConnectionException;
{code}
and:
{code}
public HBaseAdmin(HConnection connection) throws MasterNotRunningException,
ZooKeeperConnectionException;
{code}
The first constructor will retry when invoking HConnection#getMaster fail,
which performing differently as the second constructor. The second constructor
will not retry if connecting master fail. As HConnection#getMaster will also
retry if connecting fail, is it reasonable that the first constructor will not
retry when invoking HConnection#getMaster fail? Then, the two constructors will
have the same action.
The constructor of HBaseAdmin may close the shared HConnection
---
Key: HBASE-10396
URL: https://issues.apache.org/jira/browse/HBASE-10396
Project: HBase
Issue Type: Bug
Components: Admin, Client
Affects Versions: 0.94.16
Reporter: cuijianwei
Attachments: HBASE-10396-0.94-v1.patch
HBaseAdmin has the constructor:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
...
{code}
As shown in above code, HBaseAdmin will get a cached HConnection or create a
new HConnection and use this HConnection to connect to Master. Then,
HBaseAdmin will delete the HConnection when connecting to master fail as
follows:
{code}
while ( true ){
try {
this.connection.getMaster();
return;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
The above code will invoke HConnectionManager#deleteStaleConnection to delete
the HConnection from global HConnection cache. The risk is that the deleted
HConnection might be sharing by other threads, such as HTable or HTablePool.
Then, these threads which sharing the deleted HConnection will get closed
HConnection exception:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@61bc59aa
closed
{code}
If users use HTablePool, the situation will become worse because closing
HTable will only return HTable to HTablePool which won't reduce the reference
count of the closed HConnection. Then, the closed HConnection will always be
used before clearing HTablePool. In 0.94, some modules such as Rest server
are using HTablePool, therefore may suffer from this problem.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10396) The constructor of HBaseAdmin may close the shared HConnection
[
https://issues.apache.org/jira/browse/HBASE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882235#comment-13882235
]
chunhui shen commented on HBASE-10396:
--
After the patch,
{code}
catch (MasterNotRunningException mnre) {
HConnectionManager.deleteConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
It means the connection won't be changed in the while block since always get
the cached one. Is it expected?
I think we could move the HBaseAdmin code from trunk to 0.94
The constructor of HBaseAdmin may close the shared HConnection
---
Key: HBASE-10396
URL: https://issues.apache.org/jira/browse/HBASE-10396
Project: HBase
Issue Type: Bug
Components: Admin, Client
Affects Versions: 0.94.16
Reporter: cuijianwei
Attachments: HBASE-10396-0.94-v1.patch
HBaseAdmin has the constructor:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
...
{code}
As shown in above code, HBaseAdmin will get a cached HConnection or create a
new HConnection and use this HConnection to connect to Master. Then,
HBaseAdmin will delete the HConnection when connecting to master fail as
follows:
{code}
while ( true ){
try {
this.connection.getMaster();
return;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
The above code will invoke HConnectionManager#deleteStaleConnection to delete
the HConnection from global HConnection cache. The risk is that the deleted
HConnection might be sharing by other threads, such as HTable or HTablePool.
Then, these threads which sharing the deleted HConnection will get closed
HConnection exception:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@61bc59aa
closed
{code}
If users use HTablePool, the situation will become worse because closing
HTable will only return HTable to HTablePool which won't reduce the reference
count of the closed HConnection. Then, the closed HConnection will always be
used before clearing HTablePool. In 0.94, some modules such as Rest server
are using HTablePool, therefore may suffer from this problem.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10396) The constructor of HBaseAdmin may close the shared HConnection
[
https://issues.apache.org/jira/browse/HBASE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882281#comment-13882281
]
cuijianwei commented on HBASE-10396:
Thanks for your advice [~zjushch], I think moving trunk code of HBaseAdmin to
0.94 is a better choice. In patch 0.94-v1, HBaseAdmin might get a new
HConnection only when all the other threads deleted the shared HConnection,
which is not good enough. The trunk code of HBaseAdmin is:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException, IOException {
// Will not leak connections, as the new implementation of the constructor
// does not throw exceptions anymore.
this(HConnectionManager.getConnection(new Configuration(c)));
this.cleanupConnectionOnClose = true;
}
public HBaseAdmin(HConnection connection)
throws MasterNotRunningException, ZooKeeperConnectionException {
}
{code}
The constructs of HBaseAdmin in trunk will not throw exceptions. However, in
0.94, HConnection#getMaster will throw exceptions in constructor which might
cause Hconnection leak. Therefore, the second patch will not invoke
HConnection#getMaster in HBaseAdmin. Then, HConnection#getMaster will be
invoked when really needing to connect to master.
The constructor of HBaseAdmin may close the shared HConnection
---
Key: HBASE-10396
URL: https://issues.apache.org/jira/browse/HBASE-10396
Project: HBase
Issue Type: Bug
Components: Admin, Client
Affects Versions: 0.94.16
Reporter: cuijianwei
Attachments: HBASE-10396-0.94-v1.patch, HBASE-10396-0.94-v2.patch
HBaseAdmin has the constructor:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
...
{code}
As shown in above code, HBaseAdmin will get a cached HConnection or create a
new HConnection and use this HConnection to connect to Master. Then,
HBaseAdmin will delete the HConnection when connecting to master fail as
follows:
{code}
while ( true ){
try {
this.connection.getMaster();
return;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
The above code will invoke HConnectionManager#deleteStaleConnection to delete
the HConnection from global HConnection cache. The risk is that the deleted
HConnection might be sharing by other threads, such as HTable or HTablePool.
Then, these threads which sharing the deleted HConnection will get closed
HConnection exception:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@61bc59aa
closed
{code}
If users use HTablePool, the situation will become worse because closing
HTable will only return HTable to HTablePool which won't reduce the reference
count of the closed HConnection. Then, the closed HConnection will always be
used before clearing HTablePool. In 0.94, some modules such as Rest server
are using HTablePool, therefore may suffer from this problem.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10396) The constructor of HBaseAdmin may close the shared HConnection
[
https://issues.apache.org/jira/browse/HBASE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882093#comment-13882093
]
stack commented on HBASE-10396:
---
[~cuijianwei] Go for it boss.
The constructor of HBaseAdmin may close the shared HConnection
---
Key: HBASE-10396
URL: https://issues.apache.org/jira/browse/HBASE-10396
Project: HBase
Issue Type: Bug
Components: Admin, Client
Affects Versions: 0.94.16
Reporter: cuijianwei
HBaseAdmin has the constructor:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
...
{code}
As shown in above code, HBaseAdmin will get a cached HConnection or create a
new HConnection and use this HConnection to connect to Master. Then,
HBaseAdmin will delete the HConnection when connecting to master fail as
follows:
{code}
while ( true ){
try {
this.connection.getMaster();
return;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
The above code will invoke HConnectionManager#deleteStaleConnection to delete
the HConnection from global HConnection cache. The risk is that the deleted
HConnection might be sharing by other threads, such as HTable or HTablePool.
Then, these threads which sharing the deleted HConnection will get closed
HConnection exception:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@61bc59aa
closed
{code}
If users use HTablePool, the situation will become worse because closing
HTable will only return HTable to HTablePool which won't reduce the reference
count of the closed HConnection. Then, the closed HConnection will always be
used before clearing HTablePool. In 0.94, some modules such as Rest server
are using HTablePool, therefore may suffer from this problem.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10396) The constructor of HBaseAdmin may close the shared HConnection
[
https://issues.apache.org/jira/browse/HBASE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878423#comment-13878423
]
cuijianwei commented on HBASE-10396:
To alleviate this problem, we might invoke HConnection#close when connecting to
Master fail. However, this won't close the HConnection immediately if the
HConnection is sharing by other threads. Then, the same HConnection will be got
in the next retry.
The constructor of HBaseAdmin may close the shared HConnection
---
Key: HBASE-10396
URL: https://issues.apache.org/jira/browse/HBASE-10396
Project: HBase
Issue Type: Bug
Components: Admin, Client
Affects Versions: 0.94.16
Reporter: cuijianwei
HBaseAdmin has the constructor:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
...
{code}
As shown in above code, HBaseAdmin will get a cached HConnection or create a
new HConnection and use this HConnection to connect to Master. Then,
HBaseAdmin will delete the HConnection when connecting to master fail as
follows:
{code}
while ( true ){
try {
this.connection.getMaster();
return;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
The above code will invoke HConnectionManager#deleteStaleConnection to delete
the HConnection from global HConnection cache. The risk is that the deleted
HConnection might be sharing by other threads, such as HTable or HTablePool.
Then, these threads which sharing the deleted HConnection will get closed
HConnection exception:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@61bc59aa
closed
{code}
If users use HTablePool, the situation will become worse because closing
HTable will only return HTable to HTablePool which won't reduce the reference
count of the closed HConnection. Then, the closed HConnection will always be
used before clearing HTablePool. In 0.94, some modules such as Rest server
are using HTablePool, therefore may suffer from this problem.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10396) The constructor of HBaseAdmin may close the shared HConnection
[
https://issues.apache.org/jira/browse/HBASE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878436#comment-13878436
]
chunhui shen commented on HBASE-10396:
--
Trunk seems fixed this problem. Make a backport?
The constructor of HBaseAdmin may close the shared HConnection
---
Key: HBASE-10396
URL: https://issues.apache.org/jira/browse/HBASE-10396
Project: HBase
Issue Type: Bug
Components: Admin, Client
Affects Versions: 0.94.16
Reporter: cuijianwei
HBaseAdmin has the constructor:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
...
{code}
As shown in above code, HBaseAdmin will get a cached HConnection or create a
new HConnection and use this HConnection to connect to Master. Then,
HBaseAdmin will delete the HConnection when connecting to master fail as
follows:
{code}
while ( true ){
try {
this.connection.getMaster();
return;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
The above code will invoke HConnectionManager#deleteStaleConnection to delete
the HConnection from global HConnection cache. The risk is that the deleted
HConnection might be sharing by other threads, such as HTable or HTablePool.
Then, these threads which sharing the deleted HConnection will get closed
HConnection exception:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@61bc59aa
closed
{code}
If users use HTablePool, the situation will become worse because closing
HTable will only return HTable to HTablePool which won't reduce the reference
count of the closed HConnection. Then, the closed HConnection will always be
used before clearing HTablePool. In 0.94, some modules such as Rest server
are using HTablePool, therefore may suffer from this problem.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10396) The constructor of HBaseAdmin may close the shared HConnection
[
https://issues.apache.org/jira/browse/HBASE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878481#comment-13878481
]
cuijianwei commented on HBASE-10396:
Thanks for your comment [~zjushch], I go through the code of HBaseAdmin in
trunk. The HConnection will be closed in HBaseAdmin#close so that fixed the
problem. The code of HBaseAdmin changed a lot between 0.94 and trunk, will we
supply a patch for 0.94 to fix this problem?
The constructor of HBaseAdmin may close the shared HConnection
---
Key: HBASE-10396
URL: https://issues.apache.org/jira/browse/HBASE-10396
Project: HBase
Issue Type: Bug
Components: Admin, Client
Affects Versions: 0.94.16
Reporter: cuijianwei
HBaseAdmin has the constructor:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
...
{code}
As shown in above code, HBaseAdmin will get a cached HConnection or create a
new HConnection and use this HConnection to connect to Master. Then,
HBaseAdmin will delete the HConnection when connecting to master fail as
follows:
{code}
while ( true ){
try {
this.connection.getMaster();
return;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
The above code will invoke HConnectionManager#deleteStaleConnection to delete
the HConnection from global HConnection cache. The risk is that the deleted
HConnection might be sharing by other threads, such as HTable or HTablePool.
Then, these threads which sharing the deleted HConnection will get closed
HConnection exception:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@61bc59aa
closed
{code}
If users use HTablePool, the situation will become worse because closing
HTable will only return HTable to HTablePool which won't reduce the reference
count of the closed HConnection. Then, the closed HConnection will always be
used before clearing HTablePool. In 0.94, some modules such as Rest server
are using HTablePool, therefore may suffer from this problem.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
