[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438331 ]
Dawid Weiss commented on HADOOP-538:
------------------------------------
Doug is correct -- compiling native code is a pain and including it in the
runtime is even more pain. I have a project that uses JNI to bind to a shared
library, which in turn calls other libraries. Apart from setting up the build
process in the ANT build file, which is itself quite tricky -- the same set of
tools (gcc) on Linux produces *.so, whereas on Cygwin it creates *.dll files --
you need to set up java.library.path to point to your shared objects AND a
regular path (Windows) or LD_LIBRARY_PATH (Linux-ish) so that the system can
find native dependencies. In the end, it's a real pain to compile and maintain.
The suggestion to fallback to pure java is I think a good one. It allows you to
start playing with Hadoop on heterogenic environment and without much hassle
and then tune it later (by placing appropriate libraries wherever the system
can find them).
> Implement a nio's 'direct buffer' based wrapper over zlib to improve
> performance of java.util.zip.{De|In}flater as a 'custom codec'
> -----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-538
> URL: http://issues.apache.org/jira/browse/HADOOP-538
> Project: Hadoop
> Issue Type: Improvement
> Affects Versions: 0.6.1
> Reporter: Arun C Murthy
> Assigned To: Arun C Murthy
> Fix For: 0.7.0
>
> Attachments: HADOOP-538.patch, HADOOP-538_benchmarks.tgz
>
>
> There has been more than one instance where java.util.zip's {De|In}flater
> classes perform unreliably, a simple wrapper over zlib-1.2.3 (latest stable)
> using java.nio.ByteBuffer (i.e. direct buffers) should go a long way in
> alleviating these woes.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
