Yes. That is what I was trying to convey with a TezATSClient - it gives a Tez facade that provides expected semantics like getFooForVertex() etc. and hides the actual ATS details from the user.
-----Original Message----- From: Andre Kelpe [mailto:[email protected]] Sent: Thursday, December 10, 2015 2:09 AM To: [email protected] Subject: Re: api compatibility within a minor release >From our point of view ATS is an implementation detail. As you said in >Budapest, it changes all the time anyway, so it is not the API we would like >to talk to. We care about the information stored in the ATS that is relevant >for the current DAG, but we don't really care about the fact that the ATS is >used. In MapReduce you also don't know, that the information is stored int he >JobHistoryServer. There is an API on the RunningJob class to get to that >information. We thing something along those lines would be good. Then you can >change the DAGClient and the storage for the information all you want, we as >consumers never have to think about it. - André On Wed, Dec 9, 2015 at 11:39 PM, Bikas Saha <[email protected]> wrote: > Are we basically looking for detailed data that’s stored in the ATS? > Then should we consider creating a TezATSClient that understand Tez > semantics and fetches this information (in general for any DAG) > instead of augmenting DAGClient (that is responsible for submitting a > Tez DAG and monitoring a single DAG)? > > Bikas > > -----Original Message----- > From: Andre Kelpe [mailto:[email protected]] > Sent: Wednesday, December 9, 2015 10:32 AM > To: Hitesh Shah <[email protected]> > Cc: [email protected] > Subject: Re: api compatibility within a minor release > > Hi, > > (undusting this thread after a while) > > I have opened https://issues.apache.org/jira/browse/TEZ-2981 to work > around the issue. We believe that would be the best way forward. > > - André > > On Tue, Sep 15, 2015 at 11:04 PM, Hitesh Shah <[email protected]> wrote: > > > Hi Andre, > > > > I don’t believe I saw a JIRA created for this? Has this issue been > > resolved? > > > > Also, it would be good if you can collate any particular private > > APIs that you are using so that we can be aware of potential issues > > with such > APIs. > > > > thanks > > — Hitesh > > > > On Aug 20, 2015, at 2:10 AM, Andre Kelpe <[email protected]> > wrote: > > > > > Thanks for the answer. We have a work-around for now. I am going > > > to make that inventory and submit a patch that changes the > > > annotations from @Private to @LimitedPrivate. > > > > > > From a semantic versioning point of view, I would still expect no > > breaking > > > changes in a bug fix release, but if the Tez community at large > > > can work with that, we have to accept that, I guess. > > > > > > - André > > > > > > On Tue, Aug 18, 2015 at 8:34 PM, Hitesh Shah <[email protected]> > wrote: > > > > > >> Hello Andre, > > >> > > >> For the most part, @Private is considered internal implementation > > >> and subject to change at any point. In this case, even more so as > > >> it is an *Impl class. > > >> > > >> What we can do is try the following: > > >> - Look at all the various @Private classes being used by Cascading. > > >> - See which ones should not be used at all and which ones can > > >> be considered to be a @LimitedPrivate for Cascading. > > >> - For LimitedPrivate apis, we would then try to be a bit more > > >> careful with respect to changing/breaking these APIs. I would > > >> probably not say > > that > > >> they have will have the same guarantees as @Public/@Stable but we > > >> can > > work > > >> with the Cascading community to handle changes to these APIs in a > > workable > > >> manner on an ongoing basis. > > >> > > >> As for this API in question, for the short term fix, I guess a > > >> simple approach might be to introduce a backward compatible API ( > > >> I believe one which does not need the timeout param passed in )? > > >> Would you mind > > filing a > > >> jira and hopefully provide a patch too? > > >> > > >> thanks > > >> — Hitesh > > >> > > >> On Aug 18, 2015, at 8:01 AM, Andre Kelpe > > >> <[email protected]> > > wrote: > > >> > > >>> Hi, > > >>> > > >>> I have found a small API incompatibility in the Tez 0.6.2 release. > > >>> The DAGClientTimelineImpl constructor got a new parameter for > time-outs. > > This > > >>> was not present in 0.6.1 and from a semantic versioning point of > > >>> view, > > >> that > > >>> is problematic. I know that the class is marked with the > > >>> @Private annotation, but it would be great if such incompatible > > >>> things aren't introduced in a bug-fix release. It would be > > >>> easier for downsteam > > >> projects, > > >>> if you just added a second constructor. > > >>> > > >>> Is that an oversight or are classes with the @Private annotation > > subject > > >> to > > >>> change and dowstream projects simply have to deal with it? > > >>> > > >>> Thanks! > > >>> > > >>> - André > > >>> > > >>> -- > > >>> André Kelpe > > >>> [email protected] > > >>> http://concurrentinc.com > > >> > > >> > > > > > > > > > -- > > > André Kelpe > > > [email protected] > > > http://concurrentinc.com > > > > > > > -- > André Kelpe > [email protected] > http://concurrentinc.com > > -- André Kelpe [email protected] http://concurrentinc.com
