I have a hack to gather custom application metrics in a Streaming job, but I wanted to know if there is any better way of doing this.
My hack consists of this singleton: object Metriker extends Serializable { @transient lazy val mr: MetricRegistry = { val metricRegistry = new MetricRegistry() val graphiteEndpoint = new InetSocketAddress(" ec2-54-220-56-229.eu-west-1.compute.amazonaws.com", 2003) GraphiteReporter .forRegistry(metricRegistry) .build(new Graphite(graphiteEndpoint)) .start(5, TimeUnit.SECONDS) metricRegistry } @transient lazy val processId = ManagementFactory.getRuntimeMXBean.getName @transient lazy val hostId = { try { InetAddress.getLocalHost.getHostName } catch { case e: UnknownHostException => "localhost" } } def metricName(name: String): String = { "%s.%s.%s".format(name, hostId, processId) } } which I then use in my jobs like so: stream .map { i => Metriker.mr.meter(Metriker.metricName("testmetric123")).mark(i) i * 2 } Then I aggregate the metrics on Graphite. This works, but I was curious to know if anyone has a less hacky way. ᐧ