The communication is via HTTP. The code just uses the TimelineClient which by default is configured to connect to “0.0.0.0:8188” which is usually a problem on a multi-node setup :) To configure it, you need to set yarn.timeline-service.webapp.address to the host:port of the timeline server. This is both a client and server-side property in some sense as the client ( Tez ) needs it to connect to the server and the server uses it to decide which port to bind to.
I believe the yarn distributed shell application may have some support of sending data to ATS. A question that should have been asked earlier - what version of Tez are you running? — Hitesh On Nov 12, 2014, at 9:30 PM, Dharmesh Kakadia <[email protected]> wrote: > Hey before filing JIRA, I had just one small question. > > How does the tez communicate to the ATS ? via HDFS directory or via some > port? Which configuration on both ATS and Tez side controls that ? > > Also, if you know someway to test ATS, without tez, it would help narrow down > the problem. > > Thanks, > Dharmesh > > On Tue, Nov 11, 2014 at 9:51 PM, Jonathan Eagles <[email protected]> wrote: > Great, Dharmesh. Make sure to include the web address you are trying > to browse (for example localhost:8188/ws/v1/timeline/TEZ_DAG_ID). That > will help in debugging. > > Jon > > On Tue, Nov 11, 2014 at 10:17 AM, Dharmesh Kakadia <[email protected]> > wrote: > > I will file a JIRA with log attached. > > > > Currently I am only looking at the TLS web UI. Planning to do programmatic > > access if this runs. > > > > Thanks, > > Dharmesh > > > > On Tue, Nov 11, 2014 at 9:41 PM, Hitesh Shah <[email protected]> wrote: > >> > >> Hi Dharmesh > >> > >> Would you mind filing a jira and attaching the application logs obtained > >> by running “bin/yarn logs -applicationId <appId>” > >> Also, I am assuming you are setting “tez.am.log.level” to DEBUG to get > >> debug logging? > >> > >> Also, when you query the timeline server, what are the calls that you are > >> making to retrieve the data? > >> > >> thanks > >> — Hitesh > >> > >> > >> On Nov 11, 2014, at 7:41 AM, Dharmesh Kakadia <[email protected]> wrote: > >> > >> > Both timelineserver and tez was running on the same machine, so I doubt > >> > that could be problem. > >> > Anyway, I set the server address to be the hostname and still nothing on > >> > TLS. > >> > > >> > I appreciate your help. > >> > > >> > Thanks, > >> > Dharmesh > >> > > >> > On Tue, Nov 11, 2014 at 8:53 PM, Prakash Ramachandran > >> > <[email protected]> wrote: > >> > Apologies for the confusion. What I meant was the server address cannot > >> > be 0.0.0.0 it should be some IP address the client can reach to. > >> > > >> > On Nov 11, 2014 8:32 PM, "Dharmesh Kakadia" <[email protected]> wrote: > >> > Hi Prakash, > >> > > >> > I have the "yarn.timeline-service.hostname" already set to 0.0.0.0 and > >> > after suggestion from Rajesh, I have set > >> > tez.history.logging.service.class > >> > also, but TLS is still not showing anything. > >> > > >> > On a side note, is there any simple way to debug this? I am running tez > >> > with DEBUG level and there is no error I could find. > >> > > >> > Thanks, > >> > Dharmesh > >> > > >> > On Tue, Nov 11, 2014 at 8:07 PM, Prakash Ramachandran > >> > <[email protected]> wrote: > >> > you have set the yarn.timeline-service.hostname to 0.0.0.0. > >> > on the server it means bind on all ip addresses available, however from > >> > client its an invalid address. > >> > > >> > also in tez-site you will need to set the > >> > tez.history.logging.service.class property to > >> > org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService > >> > > >> > > >> > On 11/11/14 7:40 PM, Dharmesh Kakadia wrote: > >> >> I had not set that property initially. > >> >> But even after setting that property in the tez-site, nothing is > >> >> turning up in the TLS. > >> >> > >> >> Thanks, > >> >> Dharmesh > >> >> > >> >> On Tue, Nov 11, 2014 at 7:33 PM, Rajesh Balamohan > >> >> <[email protected]> wrote: > >> >> Can you check if you have set "tez.history.logging.service.class" to > >> >> "org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService" ? > >> >> > >> >> ~Rajesh.B > >> >> > >> >> On Tue, Nov 11, 2014 at 5:55 AM, Dharmesh Kakadia <[email protected]> > >> >> wrote: > >> >> Hi, > >> >> > >> >> I am trying to get Tez job statistics through TmielineServer API. > >> >> I have set up build Tez 0.5.1 and I am able to run tez jobs but no data > >> >> is being pushed to the TimelineServer. THe web interface shows nothing. > >> >> > >> >> Here is what I added to the yarn-site.xml > >> >> > >> >> <property> > >> >> <name>yarn.timeline-service.enabled</name> > >> >> <value>true</value> > >> >> </property> > >> >> > >> >> <property> > >> >> <name>yarn.timeline-service.hostname</name> > >> >> <value>0.0.0.0</value> > >> >> </property> > >> >> > >> >> <property> > >> >> > >> >> <name>yarn.timeline-service.generic-application-history.enabled</name> > >> >> <value>true</value> > >> >> </property> > >> >> > >> >> <property> > >> >> > >> >> <name>yarn.timeline-service.generic-application-history.store-class</name> > >> >> > >> >> <value>org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore</value> > >> >> </property> > >> >> > >> >> > >> >> <property> > >> >> <name>yarn.timeline-service.store-class</name> > >> >> > >> >> <value>org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore</value> > >> >> </property> > >> >> > >> >> Here is my tez-site.xml > >> >> > >> >> <configuration> > >> >> <property> > >> >> <name>tez.lib.uris</name> > >> >> <value>${fs.default.name}/user/hadoop/tez-apps/</value> > >> >> </property> > >> >> > >> >> <property> > >> >> <name>tez.yarn.ats.enabled</name> > >> >> <value>true</value> > >> >> </property> > >> >> > >> >> </configuration> > >> >> > >> >> I can not figure out what am I missing. Any help would be great. > >> >> > >> >> Thanks, > >> >> Dharmesh > >> >> > >> >> > >> >> > >> >> > >> >> -- > >> >> ~Rajesh.B > >> >> > >> > > >> > > >> > CONFIDENTIALITY NOTICE > >> > NOTICE: This message is intended for the use of the individual or entity > >> > to which it is addressed and may contain information that is > >> > confidential, > >> > privileged and exempt from disclosure under applicable law. If the > >> > reader of > >> > this message is not the intended recipient, you are hereby notified that > >> > any > >> > printing, copying, dissemination, distribution, disclosure or forwarding > >> > of > >> > this communication is strictly prohibited. If you have received this > >> > communication in error, please contact the sender immediately and delete > >> > it > >> > from your system. Thank You. > >> > > >> > > >> > CONFIDENTIALITY NOTICE > >> > NOTICE: This message is intended for the use of the individual or entity > >> > to which it is addressed and may contain information that is > >> > confidential, > >> > privileged and exempt from disclosure under applicable law. If the > >> > reader of > >> > this message is not the intended recipient, you are hereby notified that > >> > any > >> > printing, copying, dissemination, distribution, disclosure or forwarding > >> > of > >> > this communication is strictly prohibited. If you have received this > >> > communication in error, please contact the sender immediately and delete > >> > it > >> > from your system. Thank You. > >> > > >> > > >
