More background on why I brought up the issue.
Since we cannot cache hbaseadmin, we call the following code in a dedicated
thread:
try {
HBaseConfiguration _conf = new HBaseConfiguration();
HBaseAdmin _admin = new HBaseAdmin(_conf);
isConnected = true;
return _admin;
} catch (ConcurrentModificationException _e) {
_ex = _e;
}
After some time, I saw:
Exception in thread "HBase-Connector" java.lang.OutOfMemoryError: Java heap
space
at
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.createChunk(DeferredDocumentImpl.java:1921)
at
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.ensureCapacity(DeferredDocumentImpl.java:1826)
at
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.createNode(DeferredDocumentImpl.java:1840)
at
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.createDeferredTextNode(DeferredDocumentImpl.java:534)
at
com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1189)
at
com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1082)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:463)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:225)
at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:283)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1078)
at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1029)
at
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:979)
at
org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1015)
at
org.apache.hadoop.hbase.HBaseConfiguration.hashCode(HBaseConfiguration.java:63)
at java.util.WeakHashMap.put(WeakHashMap.java:401)
at
org.apache.hadoop.conf.Configuration.<init>(Configuration.java:213)
at
org.apache.hadoop.conf.Configuration.<init>(Configuration.java:197)
at
org.apache.hadoop.hbase.HBaseConfiguration.<init>(HBaseConfiguration.java:33)
at
net.ks.datastore.HBaseDataStore.getHBaseAdmin(HBaseDataStore.java:159)
at
net.ks.datastore.HBaseDataStore.isConnected(HBaseDataStore.java:476)
at
net.ks.datastore.HBaseDataStore.access$100(HBaseDataStore.java:39)
at net.ks.datastore.HBaseDataStore$1.run(HBaseDataStore.java:71)
On Mon, Mar 29, 2010 at 2:35 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> https://issues.apache.org/jira/browse/HBASE-2391 has been filed.
>
> We call HBaseAdmin.isMasterRunning() to see if client has connection with
> HBase. If there is a working alternative, we'd love to use it.
>
> Thanks J-D.
>
>
> On Mon, Mar 29, 2010 at 2:25 PM, Jean-Daniel Cryans
> <jdcry...@apache.org>wrote:
>
>> I think this method wasn't updated when we moved to Zookeeper (since
>> in pre-0.20, dead master = dead cluster), also looking at when this is
>> called, I only see it from HMerge and HBaseAdmin.isMasterRunning()...
>> which in turn isn't called anywhere in the java code (I think we call
>> it in the shell tho).
>>
>> So we should first consider if this method is useful at all, and if so
>> then what would be the best fix. For example, if you run a HMerge
>> while the master is down but the region servers are up, you're going
>> to end up with something wrong since it expects hbase to be completely
>> down.
>>
>> Can you open a jira to continue discussions there?
>>
>> Thx!
>>
>> J-D
>>
>> On Mon, Mar 29, 2010 at 2:01 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>> > Hi,
>> > While going over TableServers.isMasterRunning() in HConnectionManager, I
>> see
>> > this:
>> > public boolean isMasterRunning() {
>> > if (this.master == null) {
>> > try {
>> > getMaster();
>> >
>> > } catch (MasterNotRunningException e) {
>> > return false;
>> > }
>> > }
>> > return true;
>> > }
>> > When isMasterRunning() is called the first time, if master is obtained
>> > successfully, master field would contain reference to HMasterInterface.
>> > Subsequent calls to isMasterRunning() wouldn't throw
>> > MasterNotRunningException even if master server stops responding to
>> clients.
>> >
>> > I think master.isMasterRunning() should be called if master isn't null.
>> >
>> > Regards
>> >
>>
>
>
