Hey all
Under what circumstances will
VertexStatus vertexStatus = dagClient.getVertexStatus( getID(),
EnumSet.of( StatusGetOpts.GET_COUNTERS ) );
actually return the counters with a vertexStatus? Always, only before the
vertex is finished, only after the vertex is finished and redirection to the
history server has happened, or never.
The implication of TEZ-1495 is that this works, if I'm reading correctly.
And after a few days of mucking around, I'm under the impression I have the
timeline server configured on the cluster and in the app, and this is in a
container log:
2014-10-16 18:54:51,442 INFO
[ServiceThread:org.apache.tez.dag.history.HistoryEventHandler]
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService: Starting
ATSService
2014-10-16 18:54:51,442 DEBUG
[ServiceThread:org.apache.tez.dag.history.HistoryEventHandler]
org.apache.hadoop.service.AbstractService: Service
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl is started
But after the DAG is completed, the counters are always null.
I do see this frequently in the timeline server logs:
javax.ws.rs.WebApplicationException
at
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TimelineWebServices.getEntity(TimelineWebServices.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
And is there a JIRA open for retrieving Task level reports/counters as well?
ckw
--
Chris K Wensel
[email protected]
http://concurrentinc.com
