http://issues.apache.org/jira/browse/FLUME-1285
On Wed, Apr 10, 2013 at 4:20 PM, Israel Ekpo <[email protected]> wrote: > Brock, do you have the JIRA issue where the patches were uploaded? > > If I get some spare time this weekend I can take a shot at it. > > > > On 10 April 2013 16:44, Brock Noland <[email protected]> wrote: > >> I think we just need to copy 4-5 classes over from hadoop. Someone >> uploaded a patch I just haven't had time to work with it. >> >> >> On Wed, Apr 10, 2013 at 3:41 PM, Hari Shreedharan < >> [email protected]> wrote: >> >>> Yes. We will need to remove the format upgrade code and the old format >>> code from the file channel as well. I think it should be possible to do it, >>> but I suspect it is just that no one had the time to look at this as yet. >>> >>> -- >>> Hari Shreedharan >>> >>> On Wednesday, April 10, 2013 at 1:38 PM, Roshan Naik wrote: >>> >>> Hari, >>> i recall some discussion around the desire to get rid of this >>> dependency. i guess it means removing that piece of code that supports the >>> format upgrade ? are there any plans to do that ? >>> -roshan >>> >>> >>> On Wed, Apr 10, 2013 at 1:25 PM, Paul Chavez < >>> [email protected]> wrote: >>> >>> ** >>> I did not realize I could use pre-built binaries on Windows. Thank you, >>> I used this to grab the hadoop-common.jar from our existing Hadoop cluster >>> and there are no startup errors now. >>> >>> Thanks to you, Brock and Hari. >>> >>> -Paul >>> >>> ------------------------------ >>> *From:* Israel Ekpo [mailto:[email protected]] >>> *Sent:* Wednesday, April 10, 2013 12:07 PM >>> *To:* [email protected] >>> *Subject:* Re: FileChannel on Windows >>> >>> Paul, >>> >>> If Flume 1.3.1 is what you are looking for, you don't have to build it >>> from source. >>> >>> You can just download it directly from the site. Its already released. >>> >>> Paul Chavez <[email protected]> wrote: >>> >>> Hello, >>> >>> I've been using flume on Windows to send logs to an avro source for >>> awhile now, using the AvroClient class. I'm now trying to get a full agent >>> running to hopefully leverage the spoolDir and httpSource functionality. >>> Part of my requirements is a persistent channel so I'm attempting to get a >>> config running the utilizes the fileChannel. >>> >>> I followed the instructions at >>> http://mapredit.blogspot.com/2012/07/run-flume-13x-on-windows.html to >>> build flume, same as I did for the AvroClient. However it seems I need some >>> HDFS libraries. Can anyone point me to instructions for building the JAR(s) >>> I need to get this working? >>> >>> Currently I'm running into the following exception: >>> 10 Apr 2013 11:50:29,317 INFO [lifecycleSupervisor-1-0] >>> (org.apache.flume.channel.file.FileChannel.start:285) - Starting >>> FileChannel fc_Default { dataDirs: [c:\flume_data\log] }... >>> 10 Apr 2013 11:50:29,317 ERROR [lifecycleSupervisor-1-0] >>> (org.apache.flume.channel.file.FileChannel.start:314) - Failed to start >>> the file channel [channel=fc_Default] >>> java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:615) >>> at >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) >>> at java.net.URLClassLoader.access$000(URLClassLoader.java:58) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:197) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>> at org.apache.flume.channel.file.Log$Builder.build(Log.java:212) >>> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:302) >>> at >>> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>> at >>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.hadoop.io.Writable >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>> ... 24 more >>> 10 Apr 2013 11:50:29,317 ERROR [lifecycleSupervisor-1-0] >>> (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:238) - >>> Unable to start FileChannel fc_Default { dataDirs: [c:\flume_data\log] } - >>> Exception follows. >>> java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:615) >>> at >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) >>> at java.net.URLClassLoader.access$000(URLClassLoader.java:58) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:197) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>> at org.apache.flume.channel.file.Log$Builder.build(Log.java:212) >>> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:302) >>> at >>> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>> at >>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.hadoop.io.Writable >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>> ... 24 more >>> 10 Apr 2013 11:50:29,317 INFO [lifecycleSupervisor-1-0] >>> (org.apache.flume.channel.file.FileChannel.stop:329) - Stopping >>> FileChannel fc_Default { dataDirs: [c:\flume_data\log] }... >>> >>> >>> >>> Thank you, >>> Paul Chavez >>> >>> >>> >>> >> >> >> -- >> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org >> > > -- Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
