[
https://issues.apache.org/jira/browse/HBASE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joey Echeverria updated HBASE-1316:
-----------------------------------
Attachment: HBASE-1316-1.patch
zookeeper-native-Linux-amd64-64.tgz
zookeeper-native-headers.tgz
I've got a partial patch ready. The build relies on native-maven-plugin to
build the native code. This plugin pulls native dependencies as maven
artifacts. To make this work, I packaged up the zookeeper header files and the
static library compiled for x86-64 Linux.
In order to test the patch you need to install the artifacts into your local
maven repository. I've included a simple install.sh to do this for you. We'll
need to upload these artifacts somewhere, along with other supported
OSes/architectures in the future.
I did attempt to make both the build and runtime code work if you're not on a
supported platform, but I haven't extensively tested it.
At this point, the patch just adds support for interacting with zookeeper via
the native code. The interaction is very limited, currently only creating
ephemeral nodes is supported. One thing I did do was add a callback for the
native code to notify Java when it's session gets expired.
Right now, I'm generating my own session expiration event to send to the Java
zookeeper connection. I think this will allow the region server to shutdown if
the native session expires. It should look just like an expiration of the Java
session.
Things that are not yet implemented:
# The region server hasn't been modified to use the native code at all.
# I haven't modified the packaging part of the build. I'm not sure how we'll
want the build to generate versions of the native library for multiple
platforms.
Let me know if you think this is on the right track or if anything needs a big
rethink.
> ZooKeeper: use native threads to avoid GC stalls (JNI integration)
> ------------------------------------------------------------------
>
> Key: HBASE-1316
> URL: https://issues.apache.org/jira/browse/HBASE-1316
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.20.0
> Reporter: Andrew Purtell
> Assignee: Berk D. Demir
> Attachments: HBASE-1316-1.patch, zk_wrapper.tar.gz,
> zookeeper-native-Linux-amd64-64.tgz, zookeeper-native-headers.tgz
>
>
> From Joey Echeverria up on hbase-users@:
> We've used zookeeper in a write-heavy project we've been working on and
> experienced issues similar to what you described. After several days of
> debugging, we discovered that our issue was garbage collection. There was no
> way to guarantee we wouldn't have long pauses especially since our
> environment was the worst case for garbage collection, millions of tiny,
> short lived objects. I suspect HBase sees similar work loads frequently, if
> it's not constantly. With anything shorter than a 30 second session time out,
> we got session expiration events extremely frequently. We needed to use 60
> seconds for any real confidence that an ephemeral node disappearing meant
> something was unavailable.
> We really wanted quick recovery so we ended up writing a light-weight wrapper
> around the C API and used swig to auto-generate a JNI interface. It's not
> perfect, but since we switched to this method we've never seen a session
> expiration event and ephemeral nodes only disappear when there are network
> issues or a machine/process goes down.
> I don't know if it's worth doing the same kind of thing for HBase as it adds
> some "unnecessary" native code, but it's a solution that I found works.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
