[
https://issues.apache.org/jira/browse/HBASE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038208#comment-13038208
]
stack commented on HBASE-1316:
------------------------------
bq. We'll need to upload these artifacts somewhere, along with other supported
OSes/architectures in the future.
This should be fine. We've been doing this for various libs up to this point.
Can add this np.
Do you think this patch will be generally useful Joey? If so, maybe once its
up working in hbase, it can be contrib'd back to zk?
bq. I haven't modified the packaging part of the build. I'm not sure how we'll
want the build to generate versions of the native library for multiple
platforms.
Tell me more about this? Are you thinking we need to build the native libs
in-line with a build each time?
Do you think this feature can be optionally enabled? If we fail to load the
required native lib, do we default to old-school session handling? Or, its on
always but we only use new-style if we find the native libs?
How does this timeout relate to the zk session timeout?
{code}
+ public static int DEFAULT_HBASE_ZOOKEEPER_NATIVE_SESSION_TIMEOUT = 5000;
{code}
Thats cool that you have unit tests in place for your new methods already.
Patch so far looks great to me.
> ZooKeeper: use native threads to avoid GC stalls (JNI integration)
> ------------------------------------------------------------------
>
> Key: HBASE-1316
> URL: https://issues.apache.org/jira/browse/HBASE-1316
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.20.0
> Reporter: Andrew Purtell
> Assignee: Berk D. Demir
> Attachments: HBASE-1316-1.patch, zk_wrapper.tar.gz,
> zookeeper-native-Linux-amd64-64.tgz, zookeeper-native-headers.tgz
>
>
> From Joey Echeverria up on hbase-users@:
> We've used zookeeper in a write-heavy project we've been working on and
> experienced issues similar to what you described. After several days of
> debugging, we discovered that our issue was garbage collection. There was no
> way to guarantee we wouldn't have long pauses especially since our
> environment was the worst case for garbage collection, millions of tiny,
> short lived objects. I suspect HBase sees similar work loads frequently, if
> it's not constantly. With anything shorter than a 30 second session time out,
> we got session expiration events extremely frequently. We needed to use 60
> seconds for any real confidence that an ephemeral node disappearing meant
> something was unavailable.
> We really wanted quick recovery so we ended up writing a light-weight wrapper
> around the C API and used swig to auto-generate a JNI interface. It's not
> perfect, but since we switched to this method we've never seen a session
> expiration event and ephemeral nodes only disappear when there are network
> issues or a machine/process goes down.
> I don't know if it's worth doing the same kind of thing for HBase as it adds
> some "unnecessary" native code, but it's a solution that I found works.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
