Hi Bill,
There are two options known to me, I have tried only first one.
Apart from using profiling tools, when you want just measure one
function - there is function |time| in java, so you can measure how long
the call takes. example
<http://stackoverflow.com/questions/180158/how-do-i-time-a-methods-execution-in-java>.
Which way you take depends on your goals.
I think the answer heavily depends on your goals - would you like to
capture time for short period (debugging), log period (tuning), or for
lifetime.
For the case of |time|:
1.
The result of this function you can write into storm-logs using
storm, so you end-up with log entry like “my-function has taken 55
ms to execute” in worker log. In simple case you can just watch this
log with shell tools or storm logviewer.
However, these logs also can be processed by logstash
<http://logstash.net/>.
Logstash can send the events into elasticsearch, where, using kibana
<http://www.elasticsearch.org/overview/kibana/> you skim over them,
watch them, count them, build nice graphics, filter-out non-relevant
information and so on. It is easy to setup.
Logstash can also parse logs and send events to opentsdb (and other
metrics backends) where you can save and analyze the data. Docker
containers can be a great help
<https://registry.hub.docker.com/u/dockerfile/elasticsearch/>.
2.
For long-term/continuous measurement I would recommend to try out
storm metrics framework
<https://storm.apache.org/documentation/Metrics.html> (if you use
storm 0.9+). They are also applicable for trident. I would suggest
to use AssignableMetric or Reduced metric to calculate mean for your
function calls.
Metrics then could be processed by metrics consumer (and may end-up
in log or graphite, opentsdb or other database).
Best regards,
Michael Pershyn
On 10/27/2014 12:29 PM, Brunner, Bill wrote:
> What is the best way to capture start/end times of
> functions/aggregators in Trident? I am interested in capturing
> elapsed times for each, but would prefer not having to pass the info
> around in tuples, or write incrementally to a database. Wondering if
> anyone else has done this and how. Thanks
>
> ------------------------------------------------------------------------
> This message, and any attachments, is for the intended recipient(s)
> only, may contain information that is privileged, confidential and/or
> proprietary and subject to important terms and conditions available at
> http://www.bankofamerica.com/emaildisclaimer. If you are not the
> intended recipient, please delete this message.
