http://localhost:8188/ws/v1/timeline/apps/ is for YARN specific data which I believe needs a different setting enabled to have YARN dump all of its information to Timeline.
— Hitesh On Nov 13, 2014, at 5:28 AM, Dharmesh Kakadia <[email protected]> wrote: > Hi Prakash, > > The endpoints suggested by you is giving the counters. Thanks !! > I was only trying http://localhost:8188/ws/v1/timeline/apps/ and it was only > giving > {"entities":[]} > Thanks everyone for helping out. > > Thanks, > Dharmesh > > On Thu, Nov 13, 2014 at 6:02 PM, Prakash Ramachandran > <[email protected]> wrote: > can you include the url you are using for the webui and the rest api? > > for the rest api it should be something like > > http://localhost:8188/ws/v1/timeline/TEZ_DAG_ID/ for all dags > http://localhost:8188/ws/v1/timeline/TEZ_DAG_ID/dag_1414391213387_0105_1 for > the specific dag > > > On Thu, Nov 13, 2014 at 3:52 PM, Dharmesh Kakadia <[email protected]> wrote: > Found that this is fixed in Yarn 2.6 > (https://issues.apache.org/jira/browse/YARN-2804) which seems unrelated. > I now see the following entries in the timelineserver log, but still nothing > via webui or via rest API. > > 14/11/13 02:10:25 DEBUG timeline.TimelineDataManager: Storing the entity { > id: dag_1414391213387_0105_1, type: TEZ_DAG_ID }, JSON-style content: > {"events": > ... > > > Thanks, > Dharmesh > > > On Thu, Nov 13, 2014 at 3:01 PM, Dharmesh Kakadia <[email protected]> wrote: > Thanks Hitesh. I didn't realize that "yarn.timeline-service.webapp.address" > is also a client side configuration. May be we should document it somewhere. > HTH. > > After setting the above configuration, I am getting the following exception > in the TLS logs. > > 14/11/13 01:17:27 DEBUG timeline.TimelineDataManager: Storing entities: { id: > vertex_1414391213387_0101_1_02, type: TEZ_VERTEX_ID } > 14/11/13 01:17:27 DEBUG server.AuthenticationFilter: Request > [http://cspod317-02:8188/ws/v1/timeline/?user.name=hadoop] triggering > authentication > 14/11/13 01:17:27 DEBUG server.AuthenticationFilter: Request > [http://cspod317-02:8188/ws/v1/timeline/?user.name=hadoop] user [hadoop] > authenticated > Nov 13, 2014 1:17:27 AM > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator > buildModelAndSchemas > SEVERE: Failed to generate the schema for the JAX-B elements > com.sun.xml.bind.v2.runtime.IllegalAnnotationsException: 2 counts of > IllegalAnnotationExceptions > java.util.Map is an interface, and JAXB can't handle interfaces. > this problem is related to the following location: > at java.util.Map > at public java.util.Map > org.apache.hadoop.yarn.api.records.timeline.TimelineEvent.getEventInfo() > at org.apache.hadoop.yarn.api.records.timeline.TimelineEvent > at public java.util.List > org.apache.hadoop.yarn.api.records.timeline.TimelineEntity.getEvents() > at org.apache.hadoop.yarn.api.records.timeline.TimelineEntity > java.util.Map does not have a no-arg default constructor. > this problem is related to the following location: > at java.util.Map > at public java.util.Map > org.apache.hadoop.yarn.api.records.timeline.TimelineEvent.getEventInfo() > at org.apache.hadoop.yarn.api.records.timeline.TimelineEvent > at public java.util.List > org.apache.hadoop.yarn.api.records.timeline.TimelineEntity.getEvents() > at org.apache.hadoop.yarn.api.records.timeline.TimelineEntity > > at > com.sun.xml.bind.v2.runtime.IllegalAnnotationsException$Builder.check(IllegalAnnotationsException.java:106) > at > com.sun.xml.bind.v2.runtime.JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:489) > at > com.sun.xml.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:319) > at > com.sun.xml.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1170) > at > com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:145) > at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:248) > at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:235) > at javax.xml.bind.ContextFinder.find(ContextFinder.java:432) > at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:637) > at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:584) > at > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.buildModelAndSchemas(WadlGeneratorJAXBGrammarGenerator.java:412) > at > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.createExternalGrammar(WadlGeneratorJAXBGrammarGenerator.java:352) > at > com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:115) > at > com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104) > at > com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:120) > at > com.sun.jersey.server.impl.wadl.WadlMethodFactory$WadlOptionsMethodDispatcher.dispatch(WadlMethodFactory.java:98) > at > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) > at > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) > at > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) > at > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) > at > com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) > at > com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:886) > at > com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834) > at > com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795) > at > com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163) > at > com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) > at > com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118) > at > com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:96) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:572) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:542) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1203) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) > at > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) > at > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) > at > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > at org.mortbay.jetty.Server.handle(Server.java:326) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) > at > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) > at > org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) > at > org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) > > Nov 13, 2014 1:17:27 AM > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator > attachTypes > INFO: Couldn't find JAX-B element for class > org.apache.hadoop.yarn.server.webapp.dao.AppsInfo > > So, TLS is receiving the event but not able to store it due to the above > error. Is this version issue? I am running Tez 5.1 and our version of hadoop > derived from 2.5. > > Thanks, > Dharmesh > > > On Thu, Nov 13, 2014 at 12:10 PM, Hitesh Shah <[email protected]> wrote: > The communication is via HTTP. The code just uses the TimelineClient which by > default is configured to connect to “0.0.0.0:8188” which is usually a problem > on a multi-node setup :) > To configure it, you need to set yarn.timeline-service.webapp.address to the > host:port of the timeline server. This is both a client and server-side > property in some sense as the client ( Tez ) needs it to connect to the > server and the server uses it to decide which port to bind to. > > I believe the yarn distributed shell application may have some support of > sending data to ATS. > > A question that should have been asked earlier - what version of Tez are you > running? > > — Hitesh > > On Nov 12, 2014, at 9:30 PM, Dharmesh Kakadia <[email protected]> wrote: > > > Hey before filing JIRA, I had just one small question. > > > > How does the tez communicate to the ATS ? via HDFS directory or via some > > port? Which configuration on both ATS and Tez side controls that ? > > > > Also, if you know someway to test ATS, without tez, it would help narrow > > down the problem. > > > > Thanks, > > Dharmesh > > > > On Tue, Nov 11, 2014 at 9:51 PM, Jonathan Eagles <[email protected]> wrote: > > Great, Dharmesh. Make sure to include the web address you are trying > > to browse (for example localhost:8188/ws/v1/timeline/TEZ_DAG_ID). That > > will help in debugging. > > > > Jon > > > > On Tue, Nov 11, 2014 at 10:17 AM, Dharmesh Kakadia <[email protected]> > > wrote: > > > I will file a JIRA with log attached. > > > > > > Currently I am only looking at the TLS web UI. Planning to do programmatic > > > access if this runs. > > > > > > Thanks, > > > Dharmesh > > > > > > On Tue, Nov 11, 2014 at 9:41 PM, Hitesh Shah <[email protected]> wrote: > > >> > > >> Hi Dharmesh > > >> > > >> Would you mind filing a jira and attaching the application logs obtained > > >> by running “bin/yarn logs -applicationId <appId>” > > >> Also, I am assuming you are setting “tez.am.log.level” to DEBUG to get > > >> debug logging? > > >> > > >> Also, when you query the timeline server, what are the calls that you are > > >> making to retrieve the data? > > >> > > >> thanks > > >> — Hitesh > > >> > > >> > > >> On Nov 11, 2014, at 7:41 AM, Dharmesh Kakadia <[email protected]> > > >> wrote: > > >> > > >> > Both timelineserver and tez was running on the same machine, so I doubt > > >> > that could be problem. > > >> > Anyway, I set the server address to be the hostname and still nothing > > >> > on > > >> > TLS. > > >> > > > >> > I appreciate your help. > > >> > > > >> > Thanks, > > >> > Dharmesh > > >> > > > >> > On Tue, Nov 11, 2014 at 8:53 PM, Prakash Ramachandran > > >> > <[email protected]> wrote: > > >> > Apologies for the confusion. What I meant was the server address cannot > > >> > be 0.0.0.0 it should be some IP address the client can reach to. > > >> > > > >> > On Nov 11, 2014 8:32 PM, "Dharmesh Kakadia" <[email protected]> > > >> > wrote: > > >> > Hi Prakash, > > >> > > > >> > I have the "yarn.timeline-service.hostname" already set to 0.0.0.0 and > > >> > after suggestion from Rajesh, I have set > > >> > tez.history.logging.service.class > > >> > also, but TLS is still not showing anything. > > >> > > > >> > On a side note, is there any simple way to debug this? I am running tez > > >> > with DEBUG level and there is no error I could find. > > >> > > > >> > Thanks, > > >> > Dharmesh > > >> > > > >> > On Tue, Nov 11, 2014 at 8:07 PM, Prakash Ramachandran > > >> > <[email protected]> wrote: > > >> > you have set the yarn.timeline-service.hostname to 0.0.0.0. > > >> > on the server it means bind on all ip addresses available, however from > > >> > client its an invalid address. > > >> > > > >> > also in tez-site you will need to set the > > >> > tez.history.logging.service.class property to > > >> > org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService > > >> > > > >> > > > >> > On 11/11/14 7:40 PM, Dharmesh Kakadia wrote: > > >> >> I had not set that property initially. > > >> >> But even after setting that property in the tez-site, nothing is > > >> >> turning up in the TLS. > > >> >> > > >> >> Thanks, > > >> >> Dharmesh > > >> >> > > >> >> On Tue, Nov 11, 2014 at 7:33 PM, Rajesh Balamohan > > >> >> <[email protected]> wrote: > > >> >> Can you check if you have set "tez.history.logging.service.class" to > > >> >> "org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService" ? > > >> >> > > >> >> ~Rajesh.B > > >> >> > > >> >> On Tue, Nov 11, 2014 at 5:55 AM, Dharmesh Kakadia > > >> >> <[email protected]> > > >> >> wrote: > > >> >> Hi, > > >> >> > > >> >> I am trying to get Tez job statistics through TmielineServer API. > > >> >> I have set up build Tez 0.5.1 and I am able to run tez jobs but no > > >> >> data > > >> >> is being pushed to the TimelineServer. THe web interface shows > > >> >> nothing. > > >> >> > > >> >> Here is what I added to the yarn-site.xml > > >> >> > > >> >> <property> > > >> >> <name>yarn.timeline-service.enabled</name> > > >> >> <value>true</value> > > >> >> </property> > > >> >> > > >> >> <property> > > >> >> <name>yarn.timeline-service.hostname</name> > > >> >> <value>0.0.0.0</value> > > >> >> </property> > > >> >> > > >> >> <property> > > >> >> > > >> >> <name>yarn.timeline-service.generic-application-history.enabled</name> > > >> >> <value>true</value> > > >> >> </property> > > >> >> > > >> >> <property> > > >> >> > > >> >> <name>yarn.timeline-service.generic-application-history.store-class</name> > > >> >> > > >> >> <value>org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore</value> > > >> >> </property> > > >> >> > > >> >> > > >> >> <property> > > >> >> <name>yarn.timeline-service.store-class</name> > > >> >> > > >> >> <value>org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore</value> > > >> >> </property> > > >> >> > > >> >> Here is my tez-site.xml > > >> >> > > >> >> <configuration> > > >> >> <property> > > >> >> <name>tez.lib.uris</name> > > >> >> <value>${fs.default.name}/user/hadoop/tez-apps/</value> > > >> >> </property> > > >> >> > > >> >> <property> > > >> >> <name>tez.yarn.ats.enabled</name> > > >> >> <value>true</value> > > >> >> </property> > > >> >> > > >> >> </configuration> > > >> >> > > >> >> I can not figure out what am I missing. Any help would be great. > > >> >> > > >> >> Thanks, > > >> >> Dharmesh > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> -- > > >> >> ~Rajesh.B > > >> >> > > >> > > > >> > > > >> > CONFIDENTIALITY NOTICE > > >> > NOTICE: This message is intended for the use of the individual or > > >> > entity > > >> > to which it is addressed and may contain information that is > > >> > confidential, > > >> > privileged and exempt from disclosure under applicable law. If the > > >> > reader of > > >> > this message is not the intended recipient, you are hereby notified > > >> > that any > > >> > printing, copying, dissemination, distribution, disclosure or > > >> > forwarding of > > >> > this communication is strictly prohibited. If you have received this > > >> > communication in error, please contact the sender immediately and > > >> > delete it > > >> > from your system. Thank You. > > >> > > > >> > > > >> > CONFIDENTIALITY NOTICE > > >> > NOTICE: This message is intended for the use of the individual or > > >> > entity > > >> > to which it is addressed and may contain information that is > > >> > confidential, > > >> > privileged and exempt from disclosure under applicable law. If the > > >> > reader of > > >> > this message is not the intended recipient, you are hereby notified > > >> > that any > > >> > printing, copying, dissemination, distribution, disclosure or > > >> > forwarding of > > >> > this communication is strictly prohibited. If you have received this > > >> > communication in error, please contact the sender immediately and > > >> > delete it > > >> > from your system. Thank You. > > >> > > > >> > > > > > > > > > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader of > this message is not the intended recipient, you are hereby notified that any > printing, copying, dissemination, distribution, disclosure or forwarding of > this communication is strictly prohibited. If you have received this > communication in error, please contact the sender immediately and delete it > from your system. Thank You. >
