May you ask on the Pig mailing list? This looks like not a Flume issue, and I think the pig knowledge is a bit limited here. The only option as I know so far has Connor described.
- Alex On Jan 14, 2013, at 10:30 AM, Dongliang Sun <[email protected]> wrote: > I have followed Connor's instructions and build the pig from pig-trunk. > > But get the same error if I do not do anything else. > log4j:ERROR Could not instantiate class > [org.apache.flume.clients.log4jappender.Log4jAppender]. > java.lang.ClassNotFoundException: > org.apache.flume.clients.log4jappender.Log4jAppender > > So what should I do next? Copy the jars from $FLUME_HOME/lib to > pig-trunk/lib manually? > > > 2013/1/14 Alexander Alten-Lorenz <[email protected]> > >> Thanks, thats correct. >> >> @Dongliang: Please follow the instructions Connor gave. >> >> - Alex >> >> On Jan 14, 2013, at 8:10 AM, Connor Woodson <[email protected]> >> wrote: >> >>> He's running Pig, not Flume, but trying to get the Flume Log4jAppender >>> working. >>> >>> If you want to try to compile with the log4jappender, download Pig from >>> Github and add the following to ivy.xml, then 'ant build': >>> >>> <dependency org="org.apache.flume" name="flume-ng-clients" rev="1.3.1"/> >>> >>> >>> >>> On Sun, Jan 13, 2013 at 10:58 PM, Alexander Alten-Lorenz < >>> [email protected]> wrote: >>> >>>> Please post the flume startup line. I guess you have missed the --conf >>>> switch. >>>> >>>> - Alex >>>> >>>> On Jan 14, 2013, at 4:25 AM, 孙东亮 <[email protected]> wrote: >>>> >>>>> I'm a newbie for the Flume, and I just set up flume for a test. >>>>> I want to use the log4jappender to get log info from PIG scripts, the >>>>> log4j.properties is: >>>>> >>>>> log4j.appender.flume = >>>> org.apache.flume.clients.log4jappender.Log4jAppender >>>>> log4j.appender.flume.Hostname = localhost >>>>> log4j.appender.flume.Port = 41414 >>>>> log4j.logger.org.apache.pig=DEBUG,flume >>>>> >>>>> And for the flume.config: >>>>> >>>>> a1.sources = r1 >>>>> a1.sinks = k1 >>>>> a1.channels = c1 >>>>> >>>>> # Describe/configure the source >>>>> a1.sources.r1.type = avro >>>>> a1.sources.r1.bind = localhost >>>>> a1.sources.r1.port = 41414 >>>>> >>>>> # Describe the sink >>>>> a1.sinks.k1.type = logger >>>>> # Use a channel which buffers events in memory >>>>> a1.channels.c1.type = memory >>>>> a1.channels.c1.capacity = 1000 >>>>> a1.channels.c1.transactionCapacity = 100 >>>>> >>>>> # Bind the source and sink to the channel >>>>> a1.sources.r1.channels = c1 >>>>> a1.sinks.k1.channel = c1 >>>>> >>>>> But I got the following error when run a pig script: >>>>> java.lang.ClassNotFoundException: >>>>> org.apache.flume.clients.log4jappender.Log4jAppender >>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>>>> at java.lang.Class.forName0(Native Method) >>>>> at java.lang.Class.forName(Class.java:169) >>>>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179) >>>>> at >>>>> >>>> >> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320) >>>>> at >>>>> >>>> >> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121) >>>>> at >>>>> >>>> >> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664) >>>>> at >>>>> >>>> >> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647) >>>>> at >>>>> >>>> >> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568) >>>>> at >>>>> >>>> >> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442) >>>>> at >>>>> >>>> >> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367) >>>>> at org.apache.pig.Main.configureLog4J(Main.java:678) >>>>> at org.apache.pig.Main.run(Main.java:337) >>>>> at org.apache.pig.Main.main(Main.java:111) >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at >>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>>> at >>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>>>> log4j:ERROR Could not instantiate appender named "flume". >>>>> >>>>> Could you please help me find what I miss or where is incorrect for the >>>>> configuration. >>>>> >>>>> Thanks a lot! >>>>> Dongliang >>>> >>>> -- >>>> Alexander Alten-Lorenz >>>> http://mapredit.blogspot.com >>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF >>>> >>>> >> >> -- >> Alexander Alten-Lorenz >> http://mapredit.blogspot.com >> German Hadoop LinkedIn Group: http://goo.gl/N8pCF >> >> -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
