[
https://issues.apache.org/jira/browse/FLUME-580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated FLUME-580:
---------------------------------
Assignee: Jonathan Hsieh
Summary: Flume needs to be consistent with autodiscovery of Hadoop
compression codecs (was: Flume needs to be consistent with autodiscovery of
HadoopLZO codec)
> Flume needs to be consistent with autodiscovery of Hadoop compression codecs
> ----------------------------------------------------------------------------
>
> Key: FLUME-580
> URL: https://issues.apache.org/jira/browse/FLUME-580
> Project: Flume
> Issue Type: Bug
> Reporter: Disabled imported user
> Assignee: Jonathan Hsieh
>
> Currently, if flume notices that LZO is configured in core-site.xml it tries
> to load it. However, since it doesn't follow the same route that Hadoop does
> for locating the JAR and corresponding so files it fails to do so. Flume has
> to leverage the same logic that hadoop has embedded in bin/hadoop:
> .....
> # setup 'java.library.path' for native-hadoop code if necessary
> if [ -d "${HADOOP_HOME}/build/native" -o -d "${HADOOP_HOME}/lib/native" -o -d
> "${HADOOP_HOME}/sbin" ]; then
> JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} -Xmx32m
> ${HADOOP_JAVA_PLATFORM_OPTS} org.apache.hadoop.util.PlatformName | sed -e "s/
> /_/g"`
> ...
> so that it can transparently find LZO objects that it tries to autodiscover.
> For the time being, however, the workaround seems to be to tell flume
> explicitly where to look for LZO via
> adding the following to /usr/lib/flume/bin/flume-env.sh:
> export JAVA_LIBRARY_PATH=/usr/lib/hadoop-0.20/lib/native/Linux-amd64-64/
> export
> FLUME_CLASSPATH=/usr/lib/hadoop-0.20/lib/hadoop-lzo-20101122174751.20101122171345.552b3f9.jar
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
