slightly unrelated but I've seen the flume file source also "hang" a few times on me. Restarting solved my problem. Can't really say more since I don't have a lot of data but flume failing on you is a problem and is the reason a lot of people develop their own "custom collectors".
On a related note, what are my options for monitoring data flow in a flume pipeline ? On Fri, Jun 13, 2014 at 5:09 PM, saverio mucci <[email protected]> wrote: > Yep and it keeps queuing objects until u do not restarts the agent then it > does resume the normal behaviour. > > The problem is that is giving no errors during this period like it does > neither try to send the data :/ > Il 13/giu/2014 05:10 "Roshan Naik" <[email protected]> ha scritto: > > Are you saying that flume does not resume normal operation after a HBase >> connectivity is restored ? >> -roshan >> >> >> On Thu, Jun 12, 2014 at 3:29 AM, saverio mucci <[email protected]> >> wrote: >> >>> Hi all! >>> I'm a new user to the ML and to Flume too. >>> >>> I started using flume to make sure that all my data gets into hbase from >>> the frontend http server and i'm quite satisfied. >>> The problem i got is connected to the moments when hbase does not >>> respond correctly like a crash/or network problem then flume keep storing >>> the events but does not send them to the synk without any error in the log >>> file just increasing the size of the queue. >>> All starts to work correctly again only when i restart the flume agent >>> that then starts to flush the enqueued data. >>> >>> I'm using flume 1.4.0 >>> >>> Thank you all in advance >>> Best >>> Saverio Mucci >>> >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. > >
