The specific version used by Flume 1.3.1 is version 1.0.1 You can grab the jar file from this link below and drop it in the $FLUME_HOME/lib directory where all the other jar files are located.
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/1.0.1/hadoop-core-1.0.1.jar On 10 April 2013 15:08, Hari Shreedharan <[email protected]> wrote: > This is due to a dependency on some hadoop classes. You should drop in the > hadoop-common jar file (download the hadoop-common binary and drop this jar > in) into the flume classpath. Though this is no longer used directly, there > is code which is meant to upgrade legacy format of the file channel - so > the jar is still needed. > > Hari > > > On Wed, Apr 10, 2013 at 11:55 AM, Paul Chavez < > [email protected]> wrote: > >> ** >> Hello, >> >> I've been using flume on Windows to send logs to an avro source for >> awhile now, using the AvroClient class. I'm now trying to get a full agent >> running to hopefully leverage the spoolDir and httpSource functionality. >> Part of my requirements is a persistent channel so I'm attempting to get a >> config running the utilizes the fileChannel. >> >> I followed the instructions at >> http://mapredit.blogspot.com/2012/07/run-flume-13x-on-windows.html to >> build flume, same as I did for the AvroClient. However it seems I need some >> HDFS libraries. Can anyone point me to instructions for building the JAR(s) >> I need to get this working? >> >> Currently I'm running into the following exception: >> 10 Apr 2013 11:50:29,317 INFO [lifecycleSupervisor-1-0] >> (org.apache.flume.channel.file.FileChannel.start:285) - Starting >> FileChannel fc_Default { dataDirs: [c:\flume_data\log] }... >> 10 Apr 2013 11:50:29,317 ERROR [lifecycleSupervisor-1-0] >> (org.apache.flume.channel.file.FileChannel.start:314) - Failed to start >> the file channel [channel=fc_Default] >> java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:615) >> at >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) >> at java.net.URLClassLoader.access$000(URLClassLoader.java:58) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:197) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> at org.apache.flume.channel.file.Log$Builder.build(Log.java:212) >> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:302) >> at >> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >> at >> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >> at java.lang.Thread.run(Thread.java:662) >> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.io.Writable >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> ... 24 more >> 10 Apr 2013 11:50:29,317 ERROR [lifecycleSupervisor-1-0] >> (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:238) - >> Unable to start FileChannel fc_Default { dataDirs: [c:\flume_data\log] } - >> Exception follows. >> java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:615) >> at >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) >> at java.net.URLClassLoader.access$000(URLClassLoader.java:58) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:197) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> at org.apache.flume.channel.file.Log$Builder.build(Log.java:212) >> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:302) >> at >> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >> at >> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >> at java.lang.Thread.run(Thread.java:662) >> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.io.Writable >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> ... 24 more >> 10 Apr 2013 11:50:29,317 INFO [lifecycleSupervisor-1-0] >> (org.apache.flume.channel.file.FileChannel.stop:329) - Stopping >> FileChannel fc_Default { dataDirs: [c:\flume_data\log] }... >> >> >> >> Thank you, >> Paul Chavez >> > >
