Stephen Walsh created SPARK-2769:
------------------------------------
Summary: Ganglia Support Broken / Not working
Key: SPARK-2769
URL: https://issues.apache.org/jira/browse/SPARK-2769
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.0.0
Environment: Linux Red Hat 6.4 on Spark 1.1.0
Reporter: Stephen Walsh
Hi all,
I've build spark 1.1.0 with sbt with ganglia enabled and hadoop version 2.4.0
No issues there, spark works fine on hadoop 2.4.0 and ganglia (GraphiteSink) is
installed.
I've added the following to the metrics.properties
*.sink.graphite.class=org.apache.spark.metrics.sink.GraphiteSink
*.sink.graphite.host=HOSTNAME
*.sink.graphite.port=8649
*.sink.graphite.period=1
*.sink.graphite.prefix=aa
and I get this error message
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush(BufferedWriter.java:254)
at com.codahale.metrics.graphite.Graphite.send(Graphite.java:77)
at
com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:254)
at
com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:156)
at
com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:107)
at
com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:86)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
>From looking at the code I see the following.
val graphite: Graphite = new Graphite(new InetSocketAddress(host, port))
val reporter: GraphiteReporter = GraphiteReporter.forRegistry(registry)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.convertRatesTo(TimeUnit.SECONDS)
.prefixedWith(prefix)
.build(graphite)
Followed by
override def start() {
reporter.start(pollPeriod, pollUnit)
}
I noticed that the error fails when we first fry to send a message but nowhere
do I see graphite.connect() being called?
https://github.com/dropwizard/metrics/blob/master/metrics-graphite/src/main/java/com/codahale/metrics/graphite/Graphite.java#L62
The GraphiteBuilder doesn't call it either when creating the "reporter" object.
https://github.com/dropwizard/metrics/blob/master/metrics-graphite/src/main/java/com/codahale/metrics/graphite/GraphiteReporter.java#L113
Maybe I'm looking in the wrong area and I'm passing in the wrong values - but
very little logging has me thinking it is a bug.
Regards
Steve
--
This message was sent by Atlassian JIRA
(v6.2#6252)
