Re: Ambari Metrics Collector failing...

Thu, 07 Sep 2017 08:12:20 -0700

Is this something we need to fix for 0.4.1? Also, should I create JIRA ticket? 

On 2017-09-06 16:45, [email protected] wrote:
I'm seeing the same issue right now as well on my fresh bare metal install 
of HDP (no Metron yet), haven't dug into it further to troubleshoot.

Jon

On Wed, Sep 6, 2017, 18:22 Laurens Vets <[email protected]> wrote:


In preparation of 0.4.1-rc, I'm trying to install the current github
master and I'm running into an issue with Ambari-metrics-collector.
"Metrics Collector" seems to start, but immediately turns red again
Ambari and stops.

Any idea what might be going on or where I can start troubleshooting
this?

In /var/log/hbase-ams-master-metron1.log I see lots of:

2017-09-06 21:49:16,469 INFO  [HBase-Metrics2-1] impl.MetricsConfig:
loaded properties from hadoop-metrics2-hbase.properties
2017-09-06 21:49:16,477 INFO  [HBase-Metrics2-1]
timeline.HadoopTimelineMetricsSink: Initializing Timeline metrics sink. 
2017-09-06 21:49:16,478 INFO  [HBase-Metrics2-1]
timeline.HadoopTimelineMetricsSink: Identified hostname = metron1,
serviceName = ams-hbase
2017-09-06 21:49:16,478 INFO  [HBase-Metrics2-1]
timeline.HadoopTimelineMetricsSink: Collector Uri:
http://metron1:6188/ws/v1/timeline/metrics
2017-09-06 21:49:16,491 INFO  [HBase-Metrics2-1]
impl.MetricsSinkAdapter: Sink timeline started
2017-09-06 21:49:16,500 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: 
Scheduled snapshot period at 10 second(s).
2017-09-06 21:49:16,500 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: 
HBase metrics system started
2017-09-06 21:49:16,518 WARN  [HBase-Metrics2-1] lib.Interns: Metrics
intern cache overflow at 2011 for
MetricsSystem={MetricsSystem=MetricsInfo
Impl{name=MetricsSystem, description=MetricsSystem}, MetricsSystem
record=MetricsInfoImpl{name=MetricsSystem, description=MetricsSystem
record}}
2017-09-06 21:49:17,564 WARN
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:61181] server.NIOServerCnxn:
caught end of stream exception
EndOfStreamException: Unable to read additional data from client
sessionid 0x15e5929270d0001, likely client has closed socket
         at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
         at

org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
         at java.lang.Thread.run(Thread.java:745)
2017-09-06 21:49:17,565 INFO
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:61181] server.NIOServerCnxn:
Closed socket connection for client /10.0.0.11:
45058 which had sessionid 0x15e5929270d0001q

Additionally, I also see:

2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
timeline.HadoopTimelineMetricsSink: Initializing Timeline metrics sink. 
2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
timeline.HadoopTimelineMetricsSink: Identified hostname = metron1,
serviceName = ams-hbase
2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
timeline.HadoopTimelineMetricsSink: Collector Uri:
http://metron1:6188/ws/v1/timeline/metrics
2017-09-06 22:19:16,470 INFO  [HBase-Metrics2-1]
impl.MetricsSinkAdapter: Sink timeline started
2017-09-06 22:19:16,471 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: 
Scheduled snapshot period at 10 second(s).
2017-09-06 22:19:16,471 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: 
HBase metrics system started
2017-09-06 22:20:26,491 INFO  [timeline]
timeline.HadoopTimelineMetricsSink: Unable to connect to collector,
http://metron1:6188/ws/v1/timeline/metrics
This exceptions will be ignored for next 100 times

2017-09-06 22:20:26,491 WARN  [timeline]
timeline.HadoopTimelineMetricsSink: Unable to send metrics to collector 
by address:http://metron1:6188/ws/v1/timeline/metrics

Which ok because metrics collector isn't working...

Any ideas?

Reply via email to