I am wondering if accounting latency at a tuple level will add too much overhead. It might be better to do it on a conditional basis.
On Tue, Nov 17, 2015 at 3:48 PM, Ganelin, Ilya <[email protected]> wrote: > Specifically, I¹m interested in being able to track latencies for > individual tuples. For example, as they move through the BufferServer or > the CircularBuffer. > > On 11/17/15, 3:12 PM, "David Yan" <[email protected]> wrote: > > >Hi Ilya, > > > >The AutoMetric interface is mostly designed for application-specific > >metrics within an operator -- things like how many frauds are detected, or > >the dollar amount processed. > > > >We do have BufferServer bytes and queue tuple count (a.k.a. queue size) as > >system metrics. If you have specific metrics that pertain to BufferServer > >and CircularBuffer that you're interested in, please state them. We > >should > >probably add them to system metrics since it's probably useful in general. > > > >David > > > >On Tue, Nov 17, 2015 at 2:58 PM, Pramod Immaneni <[email protected]> > >wrote: > > > >> Some of them are available in system stats like port queue size and are > >> exposed via REST and to a stats listener that you can register via the > >>API. > >> Some of the others you mentioned can be added. > >> > >> On Tue, Nov 17, 2015 at 2:52 PM, Ganelin, Ilya < > >> [email protected]> > >> wrote: > >> > >> > Hi, all presently Apex supports surfacing metrics via the > >>AutoMetrics > >> > interface, documented here: > >> > http://docs.datatorrent.com/autometrics/ > >> > > >> > However, this only provides utility to monitor operators. I would > >>like to > >> > provide a more general latency suite - fFor example, I would like to > >> > instrument metrics around the BufferServer or the CircularQueue. Not > >> > everything in Apex is an operator, and there are other components of > >>the > >> > system around which we may want explicit measurements or metrics. Is > >>this > >> > something that could exist? I know other projects like Spark have > >>similar > >> > efforts, although they are equally underdeveloped :) > >> > ________________________________________________________ > >> > > >> > The information contained in this e-mail is confidential and/or > >> > proprietary to Capital One and/or its affiliates and may only be used > >> > solely in performance of work or services for Capital One. The > >> information > >> > transmitted herewith is intended only for use by the individual or > >>entity > >> > to which it is addressed. If the reader of this message is not the > >> intended > >> > recipient, you are hereby notified that any review, retransmission, > >> > dissemination, distribution, copying or other use of, or taking of any > >> > action in reliance upon this information is strictly prohibited. If > >>you > >> > have received this communication in error, please contact the sender > >>and > >> > delete the material from your computer. > >> > > >> > > ________________________________________________________ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. > >
