Okay. I’ll take a look. From: Jeff Lord [mailto:[email protected]] Sent: Monday, January 26, 2015 5:32 PM To: [email protected] Subject: Re: Monitoring the progress of events
You should be able to use the channelsize On Mon, Jan 26, 2015 at 2:29 PM, Carlotta Hicks <[email protected]<mailto:[email protected]>> wrote: Are these the counters from MonitoredCounterGroup? What is the scope of these counters? Can you reset these counters? -----Original Message----- From: Joey Echeverria [mailto:[email protected]<mailto:[email protected]>] Sent: Monday, January 26, 2015 5:20 PM To: [email protected]<mailto:[email protected]> Subject: Re: Monitoring the progress of events Assuming you're using the AvroSource, the application will have to do something like this: 1. Send all of the events to the source 2. Send a specially crafted canary event to the source 3. Monitor the destination where the sink writes looking for the canary event You might also be able to monitor the counters that the agent maintains to see if the expected number of events have been processed by the sink. -Joey On Mon, Jan 26, 2015 at 2:17 PM, Carlotta Hicks <[email protected]<mailto:[email protected]>> wrote: > I've created a simple agent with a source, a channel and a sink. How can an > application know when all events sent to the source have been processed by > the sink? > > -----Original Message----- > From: Joey Echeverria [mailto:[email protected]<mailto:[email protected]>] > Sent: Monday, January 26, 2015 5:06 PM > To: [email protected]<mailto:[email protected]> > Subject: Re: Monitoring the progress of events > > @Seshu: If you emailed > [email protected]<mailto:[email protected]> > and it still didn't unsubscribe you you should file a JIRA with Apache's > infrastructure team: > > https://issues.apache.org/jira/browse/INFRA/?selectedTab=com.atlassian > .jira.jira-projects-plugin:summary-panel > > @Carlotta: > > What kind of monitoring are you looking for? I don't believe there is > event-level monitoring as this would be very expensive. What I've seen some > folks do is send a canary event to verify when processing of some batch has > completed. This won't work if you have fan out/fan in in the flow as the > ordering between multiple Flume agent's isn't guaranteed. > > -Joey > > On Mon, Jan 26, 2015 at 12:52 PM, Seshu V > <[email protected]<mailto:[email protected]>> wrote: >> Please take my id off of this group. Unsubscribe doesn't seem to be >> working. >> >> thanks. >> >> On Mon, Jan 26, 2015 at 2:48 PM, Carlotta Hicks >> <[email protected]<mailto:[email protected]>> >> wrote: >>> >>> Is there a way to monitor the progress of events from the time they >>> are sent to the agent until they have been processed by the sink? >>> Is there a status for an agent? >>> >>> >> >> > > > > -- > Joey Echeverria -- Joey Echeverria
