[
https://issues.apache.org/jira/browse/HADOOP-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15066916#comment-15066916
]
Colin Patrick McCabe commented on HADOOP-11127:
-----------------------------------------------
Thanks for thinking about this, [~cnauroth].
I'm concerned that if we implement fallback logic like you described, people
will experience weird behaviors and heisenbugs when the wrong libhadoop version
is installed. I think most admins would rather see a simple and clear warning
message that libhadoop-2.8.3 is not installed, than see weird behaviors like
one native function working and another not. Not all changes in the native
library general clear error messages either... something like changing an
uint32_t to a uint64_t could easily lead to crashes or buffer overflows rather
than a "dlsym failed" error message.
I think we should check only for the specific version of the library that ought
to be installed, not for any other. After all, we already have a fallback mode
where we run without JNI. If installing the correct libhadoop versions is too
much of a burden, admins can run in that mode. We just don't have the testing
bandwidth to make sure an arbitrary number of fallback configurations work as
advertised (or even fail cleanly as advertised.)
I agree that we need to handle Windows, and that we should consult with
BigTop-- those are good points.
> Improve versioning and compatibility support in native library for downstream
> hadoop-common users.
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-11127
> URL: https://issues.apache.org/jira/browse/HADOOP-11127
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: native
> Reporter: Chris Nauroth
> Assignee: Alan Burlison
> Attachments: HADOOP-11064.003.patch, proposal.01.txt
>
>
> There is no compatibility policy enforced on the JNI function signatures
> implemented in the native library. This library typically is deployed to all
> nodes in a cluster, built from a specific source code version. However,
> downstream applications that want to run in that cluster might choose to
> bundle a hadoop-common jar at a different version. Since there is no
> compatibility policy, this can cause link errors at runtime when the native
> function signatures expected by hadoop-common.jar do not exist in
> libhadoop.so/hadoop.dll.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
