yeah, you right. After successfully build the project through mvn package -DskipTest.
I have got two directories in flume source: flume-ng-dist/target/ apache-flume-1.2.0-dist.tar and apache-flume-1.2.0-src.tar. I have copied the apache-flume-1.2.0-dist.tar and unpacked into d:\apache-flume. Thanks On Mon, Jul 30, 2012 at 3:58 PM, alo alt <[email protected]> wrote: > Ah, okay. > > For non-cygwin: > > You have done git-checkout and build flume successfully (mvn -DSkipTests)? > > After you've build you've copied the binaries, located in flume's source > directory (flume-ng-dist\target\flume-ng-dist-1.3.0-SNAPSHOT-dist.tar.gz) > onto another directory and unpacked them? And that directory is > apache-flume? > > cheers, > Alex > > > > mardan Khan <[email protected]> > July 30, 2012 4:44 PM > HI, > > Actually i am doing conversation with Inder regarding the Cygwin. As I am > trying to run on cygwin and with you on MS-DOS command prompt. > > I am just telling you important location > > > 1). My flume installation directory: D:\apache-flume > 2). Lib path: D:\apache-flume\lib > 3). Configuration File: > D:\apache-flume\conf\flume-conf.properties.template (I have copied the > configuration file from your previous post). > 4). My Java path: D:\program > files\java\jdk1.7.0_03 > > My Command: > > D:\apache-flume>"c:\Program Files\Java\jdk1.7.0_03\bin\java.exe" -Xmx20m > -Dlog4j > > .configuration=file:///%CD%\conf\log4j.properties -cp > "d:\apache-flume\lib\*" org.apache.flume.node.Application > > -f d:\apache-flume\conf\flume-conf.properties.template -n syslog-agent > > > > Error: Could not find or load main class > d:\apache-flume\lib\asynchbase-1.2.0.jar > > > Now please let me know where I am doing mistake. > > > > > alo alt <[email protected]> > July 30, 2012 4:20 PM > I think we have two different conversations about the same topic. I reply > here. > > Thats not an issue of a env script in windows, because we haven't one so > far. You may need to cp (Classpath) the flume - libs as Inder says. > As I wrote in my howto: > > As example, starting from inside of your flume-installation (the directory > where you unzip'ed the build), lets assume D:\apache-flume: > d:\apache flume > "PATH\to\java.exe" -Xmx20m > -Dlog4j.configuration=file:///%CD%\conf\log4j.properties > -cp "d:\apache-flume\flume-1.3.0-SNAPSHOT\lib\*" > org.apache.flume.node.Application -f > d:\flume\flume-1.3.0-SNAPSHOT\conf\YOUR_CONFIG.conf > -n agent > > Please be sure, that the lib-directory from flume is in place as well in > the CLASSPATH (cp). > > - Alex > Inder Pall <[email protected]> > July 30, 2012 4:02 PM > That's the problem. I am not well versed with this script to fix on WIN > but i guess as a workaround you might want to add $FLUME_INSTALL_DIR/lib/. > this lib has all the required jars and should get you moving forward. > > Mind, filing a JIRA for the same. > > - inder > > > > > -- > Thanks, > - Inder > Tech Platforms @Inmobi > Linkedin - http://goo.gl/eR4Ub > mardan Khan <[email protected]> > July 30, 2012 3:53 PM > Thanks Inder, > > when I am running the $echo $CLASSPATH, I am just getting . (dot) and > nothing more. > > Please see below the flume-ng files. Can you just pointout where I should > put echo$CLASSPATH. > > Thanks > > > CLEAN_FLAG=1 > ################################ > # functions > ################################ > > info() { > if [ ${CLEAN_FLAG} -ne 0 ]; then > local msg=$1 > echo "Info: $msg" >&2 > fi > } > > warn() { > if [ ${CLEAN_FLAG} -ne 0 ]; then > local msg=$1 > echo "Warning: $msg" >&2 > fi > } > > error() { > local msg=$1 > local exit_code=$2 > > echo "Error: $msg" >&2 > > if [ -n "$exit_code" ] ; then > exit $exit_code > fi > } > > # If avail, add Hadoop paths to the FLUME_CLASSPATH and to the > # FLUME_JAVA_LIBRARY_PATH env vars. > # Requires Flume jars to already be on FLUME_CLASSPATH. > add_hadoop_paths() { > local HADOOP_IN_PATH=$(PATH="${HADOOP_HOME:-${HADOOP_PREFIX}}/bin:$PATH" > \ > which hadoop 2>/dev/null) > > if [ -f "${HADOOP_IN_PATH}" ]; then > info "Including Hadoop libraries found via ($HADOOP_IN_PATH) for HDFS > access" > > # determine hadoop java.library.path and use that for flume > local HADOOP_CLASSPATH="" > local HADOOP_JAVA_LIBRARY_PATH=$(HADOOP_CLASSPATH="$FLUME_CLASSPATH" \ > ${HADOOP_IN_PATH} org.apache.flume.tools.GetJavaProperty \ > java.library.path 2>/dev/null) > > # look for the line that has the desired property value > # (considering extraneous output from some GC options that write to > stdout) > # IFS = InternalFieldSeparator (set to recognize only newline char as > delimiter) > IFS=$'\n' > for line in $HADOOP_JAVA_LIBRARY_PATH; do > if [[ $line =~ ^java\.library\.path=(.*)$ ]]; then > HADOOP_JAVA_LIBRARY_PATH=${BASH_REMATCH[1]} > break > fi > done > unset IFS > > if [ -n "${HADOOP_JAVA_LIBRARY_PATH}" ]; then > > FLUME_JAVA_LIBRARY_PATH="$FLUME_JAVA_LIBRARY_PATH:$HADOOP_JAVA_LIBRARY_PATH" > fi > > # determine hadoop classpath > HADOOP_CLASSPATH=$($HADOOP_IN_PATH classpath 2>/dev/null) > > # hack up and filter hadoop classpath > local ELEMENTS=$(sed -e 's/:/ /g' <<<${HADOOP_CLASSPATH}) > local ELEMENT > for ELEMENT in $ELEMENTS; do > local PIECE > for PIECE in $(echo $ELEMENT); do > if [[ $PIECE =~ slf4j-(api|log4j12).*\.jar ]]; then > info "Excluding $PIECE from classpath" > continue > else > FLUME_CLASSPATH="$FLUME_CLASSPATH:$PIECE" > fi > done > done > > fi > } > add_HBASE_paths() { > local HBASE_IN_PATH=$(PATH="${HBASE_HOME}/bin:$PATH" \ > which hbase 2>/dev/null) > > if [ -f "${HBASE_IN_PATH}" ]; then > info "Including HBASE libraries found via ($HBASE_IN_PATH) for HBASE > access" > > # determine HBASE java.library.path and use that for flume > local HBASE_CLASSPATH="" > local HBASE_JAVA_LIBRARY_PATH=$(HBASE_CLASSPATH="$FLUME_CLASSPATH" \ > ${HBASE_IN_PATH} org.apache.flume.tools.GetJavaProperty \ > java.library.path 2>/dev/null) > > # look for the line that has the desired property value > # (considering extraneous output from some GC options that write to > stdout) > # IFS = InternalFieldSeparator (set to recognize only newline char as > delimiter) > IFS=$'\n' > for line in $HBASE_JAVA_LIBRARY_PATH; do > if [[ $line =~ ^java\.library\.path=(.*)$ ]]; then > HBASE_JAVA_LIBRARY_PATH=${BASH_REMATCH[1]} > break > fi > done > unset IFS > > if [ -n "${HBASE_JAVA_LIBRARY_PATH}" ]; then > > FLUME_JAVA_LIBRARY_PATH="$FLUME_JAVA_LIBRARY_PATH:$HBASE_JAVA_LIBRARY_PATH" > fi > > # determine HBASE classpath > HBASE_CLASSPATH=$($HBASE_IN_PATH classpath 2>/dev/null) > > # hack up and filter HBASE classpath > local ELEMENTS=$(sed -e 's/:/ /g' <<<${HBASE_CLASSPATH}) > local ELEMENT > for ELEMENT in $ELEMENTS; do > local PIECE > for PIECE in $(echo $ELEMENT); do > if [[ $PIECE =~ slf4j-(api|log4j12).*\.jar ]]; then > info "Excluding $PIECE from classpath" > continue > else > FLUME_CLASSPATH="$FLUME_CLASSPATH:$PIECE" > fi > done > done > FLUME_CLASSPATH="$FLUME_CLASSPATH:$HBASE_HOME/conf" > > fi > } > > display_help() { > cat <<EOF > Usage: $0 <command> [options]... > > commands: > help display this help text > agent run a Flume agent > avro-client run an avro Flume client > version show Flume version info > > global options: > --conf,-c <conf> use configs in <conf> directory > --classpath,-C <cp> append to the classpath > --dryrun,-d do not actually start Flume, just print the command > -Dproperty=value sets a JDK system property value > > agent options: > --conf-file,-f <file> specify a config file (required) > --name,-n <name> the name of this agent (required) > --help,-h display help text > > avro-client options: > --host,-H <host> hostname to which events will be sent (required) > --port,-p <port> port of the avro source (required) > --filename,-F <file> text file to stream to avro source [default: std > input] > --headerFile,-R <file> headerFile containing headers as key/value pairs > on each new line > --help,-h display help text > > Note that if <conf> directory is specified, then it is always included > first > in the classpath. > > EOF > } > > run_flume() { > local FLUME_APPLICATION_CLASS > > if [ "$#" -gt 0 ]; then > FLUME_APPLICATION_CLASS=$1 > shift > else > error "Must specify flume application class" 1 > fi > > if [ ${CLEAN_FLAG} -ne 0 ]; then > set -x > fi > $EXEC $JAVA_HOME/bin/java $JAVA_OPTS -cp "$FLUME_CLASSPATH" \ > -Djava.library.path=$FLUME_JAVA_LIBRARY_PATH > "$FLUME_APPLICATION_CLASS" $* > } > > ################################ > # main > ################################ > > # set default params > FLUME_CLASSPATH="" > FLUME_JAVA_LIBRARY_PATH="" > JAVA_OPTS="-Xmx20m" > > opt_conf="" > opt_classpath="" > opt_java_props="" > opt_dryrun="" > > mode=$1 > shift > > case "$mode" in > help) > display_help > exit 0 > ;; > agent) > opt_agent=1 > ;; > node) > opt_agent=1 > warn "The \"node\" command is deprecated. Please use \"agent\" > instead." > ;; > avro-client) > opt_avro_client=1 > ;; > version) > opt_version=1 > CLEAN_FLAG=0 > ;; > *) > error "Unknown or unspecified command '$mode'" > echo > display_help > exit 1 > ;; > esac > > while [ -n "$*" ] ; do > arg=$1 > shift > > case "$arg" in > --conf|-c) > [ -n "$1" ] || error "Option --conf requires an argument" 1 > opt_conf=$1 > shift > ;; > --classpath|-C) > [ -n "$1" ] || error "Option --classpath requires an argument" 1 > opt_classpath=$1 > shift > ;; > --dryrun|-d) > opt_dryrun="1" > ;; > -D*) > opt_java_props="$opt_java_props $arg" > ;; > *) > args="$args $arg" > ;; > esac > done > > # make opt_conf absolute > if [[ -n "$opt_conf" && -d "$opt_conf" ]]; then > opt_conf=$(cd $opt_conf; pwd) > fi > > # allow users to override the default env vars via conf/flume-env.sh > if [ -z "$opt_conf" ]; then > warn "No configuration directory set! Use --conf <dir> to override." > elif [ -f "$opt_conf/flume-env.sh" ]; then > info "Sourcing environment configuration script $opt_conf/flume-env.sh" > source "$opt_conf/flume-env.sh" > fi > > # append command-line java options to stock or env script JAVA_OPTS > if [ -n "${opt_java_props}" ]; then > JAVA_OPTS="${JAVA_OPTS} ${opt_java_props}" > fi > > # prepend command-line classpath to env script classpath > if [ -n "${opt_classpath}" ]; then > if [ -n "${FLUME_CLASSPATH}" ]; then > FLUME_CLASSPATH="${opt_classpath}:${FLUME_CLASSPATH}" > else > FLUME_CLASSPATH="${opt_classpath}" > fi > fi > > if [ -z "${FLUME_HOME}" ]; then > FLUME_HOME=$(cd $(dirname $0)/..; pwd) > fi > > # prepend $FLUME_HOME/lib jars to the specified classpath (if any) > if [ -n "${FLUME_CLASSPATH}" ] ; then > FLUME_CLASSPATH="${FLUME_HOME}/lib/*:$FLUME_CLASSPATH" > else > FLUME_CLASSPATH="${FLUME_HOME}/lib/*" > fi > > # find java > if [ -z "${JAVA_HOME}" ] ; then > warn "JAVA_HOME is not set!" > # Try to use Bigtop to autodetect JAVA_HOME if it's available > if [ -e /usr/libexec/bigtop-detect-javahome ] ; then > . /usr/libexec/bigtop-detect-javahome > elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ] ; then > . /usr/lib/bigtop-utils/bigtop-detect-javahome > fi > > # Using java from path if bigtop is not installed or couldn't find it > if [ -z "${JAVA_HOME}" ] ; then > JAVA_DEFAULT=$(type -p java) > [ -n "$JAVA_DEFAULT" ] || error "Unable to find java executable. Is it > in your PATH?" 1 > JAVA_HOME=$(cd $(dirname $JAVA_DEFAULT)/..; pwd) > fi > fi > > # look for hadoop libs > add_hadoop_paths > add_HBASE_paths > > # prepend conf dir to classpath > if [ -n "$opt_conf" ]; then > FLUME_CLASSPATH="$opt_conf:$FLUME_CLASSPATH" > fi > > # allow dryrun > EXEC="exec" > if [ -n "${opt_dryrun}" ]; then > warn "Dryrun mode enabled (will not actually initiate startup)" > EXEC="echo" > fi > > # finally, invoke the appropriate command > if [ -n "$opt_agent" ] ; then > run_flume $FLUME_AGENT_CLASS $args > elif [ -n "$opt_avro_client" ] ; then > run_flume $FLUME_AVRO_CLIENT_CLASS $args > elif [ -n "${opt_version}" ] ; then > run_flume $FLUME_VERSION_CLASS $args > else > error "This message should never appear" 1 > fi > > exit 0 > > > > > > > > > Inder Pall <[email protected]> > July 30, 2012 3:40 PM > Hey if you edit flume-ng script and just before java is invoked put an > > echo $CLASSPATH that should help in figuring what all dir's are there in > CLASSPATH. > I am sorry but i am a big fan of windows hence don't run that O.S. to try > it out. > > Thanks, > - inder > > > > > -- > Thanks, > - Inder > Tech Platforms @Inmobi > Linkedin - http://goo.gl/eR4Ub > >
<<postbox-contact.jpg>>
<<postbox-contact.jpg>>
<<compose-unknown-contact.jpg>>
