Hi, Can you please help me on one thing! Mistakenly deleted /etc/init.d/flume-master script. Now flume-master is not getting uninstalled or removed! Please help!
Thanks and Regards, Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 From: Nitin Pawar [mailto:[email protected]] Sent: Tuesday, December 11, 2012 12:55 PM To: Haldar, Shouvanik Cc: [email protected] Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1 there is no master/slave architecture now its a simple conf file like below <@EOM> and then just start the agent. it will do needful you need to start flume like the command below bin/flume-ng agent --conf-file example.conf --name agent1 agent1.sources = tail agent1.channels = MemoryChannel-2 agent1.sinks = HDFS agent1.sources.tail.type = exec agent1.sources.tail.command = tail -F <file> agent1.sources.tail.channels = MemoryChannel-2 agent1.sources.tail.interceptors = hostint agent1.sources.tail.interceptors.hostint.type = org.apache.flume.interceptor.HostInterceptor$Builder agent1.sources.tail.interceptors.hostint.preserveExisting = true agent1.sources.tail.interceptors.hostint.useIP = true agent1.sinks.HDFS.channel = MemoryChannel-2 agent1.sinks.HDFS.type = hdfs agent1.sinks.HDFS.hdfs.path = hdfs://url:8020/user/flume/%{host} agent1.sinks.HDFS.hdfs.file.Type = SequenceFile agent1.sinks.HDFS.hdfs.writeFormat = Text agent1.sinks.HDFS.hdfs.rollInterval = 30 agent1.sinks.HDFS.hdfs.rollCount = 0 agent1.sinks.HDFS.hdfs.rollSize = 5000000000 agent1.channels.MemoryChannel-2.type = memory more examples at http://flume.apache.org/FlumeUserGuide.html On Tue, Dec 11, 2012 at 12:44 PM, <[email protected]<mailto:[email protected]>> wrote: Hi, I installed flume 1.x higher version today! But is this very much different from the earlier. ? Is there no master / node concept here? How do I configure the source and sink? Think I got screwed here! Thanks and Regards, Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568> From: Nitin Pawar [mailto:[email protected]<mailto:[email protected]>] Sent: Monday, December 10, 2012 3:34 PM To: Haldar, Shouvanik Cc: [email protected]<mailto:[email protected]> Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1 just a random question, does the user running flume agent have permission to read the event details out of terracotta ? from the code here http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera/flume-core/0.9.4-cdh4.1.0/com/cloudera/flume/watchdog/Watchdog.java line 209 onwards, it looks like the error condition is achieved when you have channel set as stdin (I am not sure about this so you may need to look at code which part of code you are hitting depending on the flume configs you have) On Mon, Dec 10, 2012 at 3:01 PM, <[email protected]<mailto:[email protected]>> wrote: Hello, I am sure, the problem is due to Terracotta. Can you please help me? Thanks and Regards, Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568> From: Nitin Pawar [mailto:[email protected]<mailto:[email protected]>] Sent: Monday, December 10, 2012 2:43 PM To: Haldar, Shouvanik Cc: [email protected]<mailto:[email protected]> Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1 I just hope you are not hitting this https://issues.apache.org/jira/browse/FLUME-796 I would also like to recommend you to upgraded to flume 1.x asap if possible can you try changing your flume source to some random file with tail -f and try sinking it to some file. That will narrow the problem is on flume side or terracotta side Thanks, Nitin On Mon, Dec 10, 2012 at 1:47 PM, <[email protected]<mailto:[email protected]>> wrote: Hi, I am using Flume -> Flume 0.9.4-cdh3u5. Initially, the logs are ok. But when I start terracotta (./start-tc-server.sh) , then only I experience the above phenomena! Is it something to do with terracotta.! 2012-12-10 03:14:08,806 INFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking thread pool server on port 35853... 2012-12-10 03:14:08,894 INFO net.sf.ehcache.terracotta.TerracottaClient: Thread [logicalNode ec2-75-101-165-36.compute-1.amazonaws.com-19] [cacheManager: 'RTLogCacheManager']: Creating new ClusteredInstanceFactory Before starting terracotta, Flume Node status is "OPENING". But when I start terracotta, I got the error "INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1" Please help ! Thanks and Regards, Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568> From: Nitin Pawar [mailto:[email protected]<mailto:[email protected]>] Sent: Monday, December 10, 2012 12:52 PM To: [email protected]<mailto:[email protected]> Cc: Haldar, Shouvanik Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1 what version of flume are you using? If it is < 1.x then recommend you to upgrade to latest 1.0.3 If you are on 1.x then can you tell us whats the hostname for your server? On Mon, Dec 10, 2012 at 11:40 AM, <[email protected]<mailto:[email protected]>> wrote: Before the below mentioned error, I get a success message like 2012-12-10 01:09:14,403 INFO net.sf.ehcache.terracotta.TerracottaClient: Thread [logicalNode ec2-75-101-165-36.compute-1.amazonaws.com-19] [cacheManager: 'RTLogCacheManager']: Creating new ClusteredInstanceFactory Thanks and Regards, Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568> From: Haldar, Shouvanik Sent: Monday, December 10, 2012 11:34 AM To: '[email protected]<mailto:[email protected]>' Subject: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1 I had configured flume-master as well as flume-node on a single machine. Now, as I run terracotta -> L1 cache, I get the INFO below in / 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1 Can you please help? I am struggling with this for the past 1 day without success! Thanks and Regards, Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568> ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com<http://www.accenture.com> -- Nitin Pawar -- Nitin Pawar -- Nitin Pawar -- Nitin Pawar
