>From our point of view ATS is an implementation detail. As you said in Budapest, it changes all the time anyway, so it is not the API we would like to talk to. We care about the information stored in the ATS that is relevant for the current DAG, but we don't really care about the fact that the ATS is used. In MapReduce you also don't know, that the information is stored int he JobHistoryServer. There is an API on the RunningJob class to get to that information. We thing something along those lines would be good. Then you can change the DAGClient and the storage for the information all you want, we as consumers never have to think about it.
- André On Wed, Dec 9, 2015 at 11:39 PM, Bikas Saha <[email protected]> wrote: > Are we basically looking for detailed data that’s stored in the ATS? Then > should we consider creating a TezATSClient that understand Tez semantics > and fetches this information (in general for any DAG) instead of augmenting > DAGClient (that is responsible for submitting a Tez DAG and monitoring a > single DAG)? > > Bikas > > -----Original Message----- > From: Andre Kelpe [mailto:[email protected]] > Sent: Wednesday, December 9, 2015 10:32 AM > To: Hitesh Shah <[email protected]> > Cc: [email protected] > Subject: Re: api compatibility within a minor release > > Hi, > > (undusting this thread after a while) > > I have opened https://issues.apache.org/jira/browse/TEZ-2981 to work > around the issue. We believe that would be the best way forward. > > - André > > On Tue, Sep 15, 2015 at 11:04 PM, Hitesh Shah <[email protected]> wrote: > > > Hi Andre, > > > > I don’t believe I saw a JIRA created for this? Has this issue been > > resolved? > > > > Also, it would be good if you can collate any particular private APIs > > that you are using so that we can be aware of potential issues with such > APIs. > > > > thanks > > — Hitesh > > > > On Aug 20, 2015, at 2:10 AM, Andre Kelpe <[email protected]> > wrote: > > > > > Thanks for the answer. We have a work-around for now. I am going to > > > make that inventory and submit a patch that changes the annotations > > > from @Private to @LimitedPrivate. > > > > > > From a semantic versioning point of view, I would still expect no > > breaking > > > changes in a bug fix release, but if the Tez community at large can > > > work with that, we have to accept that, I guess. > > > > > > - André > > > > > > On Tue, Aug 18, 2015 at 8:34 PM, Hitesh Shah <[email protected]> > wrote: > > > > > >> Hello Andre, > > >> > > >> For the most part, @Private is considered internal implementation > > >> and subject to change at any point. In this case, even more so as > > >> it is an *Impl class. > > >> > > >> What we can do is try the following: > > >> - Look at all the various @Private classes being used by Cascading. > > >> - See which ones should not be used at all and which ones can be > > >> considered to be a @LimitedPrivate for Cascading. > > >> - For LimitedPrivate apis, we would then try to be a bit more > > >> careful with respect to changing/breaking these APIs. I would > > >> probably not say > > that > > >> they have will have the same guarantees as @Public/@Stable but we > > >> can > > work > > >> with the Cascading community to handle changes to these APIs in a > > workable > > >> manner on an ongoing basis. > > >> > > >> As for this API in question, for the short term fix, I guess a > > >> simple approach might be to introduce a backward compatible API ( I > > >> believe one which does not need the timeout param passed in )? > > >> Would you mind > > filing a > > >> jira and hopefully provide a patch too? > > >> > > >> thanks > > >> — Hitesh > > >> > > >> On Aug 18, 2015, at 8:01 AM, Andre Kelpe <[email protected]> > > wrote: > > >> > > >>> Hi, > > >>> > > >>> I have found a small API incompatibility in the Tez 0.6.2 release. > > >>> The DAGClientTimelineImpl constructor got a new parameter for > time-outs. > > This > > >>> was not present in 0.6.1 and from a semantic versioning point of > > >>> view, > > >> that > > >>> is problematic. I know that the class is marked with the @Private > > >>> annotation, but it would be great if such incompatible things > > >>> aren't introduced in a bug-fix release. It would be easier for > > >>> downsteam > > >> projects, > > >>> if you just added a second constructor. > > >>> > > >>> Is that an oversight or are classes with the @Private annotation > > subject > > >> to > > >>> change and dowstream projects simply have to deal with it? > > >>> > > >>> Thanks! > > >>> > > >>> - André > > >>> > > >>> -- > > >>> André Kelpe > > >>> [email protected] > > >>> http://concurrentinc.com > > >> > > >> > > > > > > > > > -- > > > André Kelpe > > > [email protected] > > > http://concurrentinc.com > > > > > > > -- > André Kelpe > [email protected] > http://concurrentinc.com > > -- André Kelpe [email protected] http://concurrentinc.com
