> Moving to 2.x means that it's my fault as a consumer of the API if I was > still using the deprecated endpoint (and that change should be part of a > larger upgrade plan). Simply removing the deprecated routes, retroactively, > means I can't rely on having a transition period to get to 2.0 outside of a > large completely down upgrade window.
There would be a transition period either way. Endpoints would be deprecated in ATC 4.0 and removed in 5.0. The release notes for 4.0 would clearly list out the deprecated endpoints. If they have replacements, there would be instructions on how to transition to the replacement. Deprecated endpoints would return alerts. After upgrading to 4.0, operators should be well aware that certain endpoints are being removed in 5.0, and it's on them to start using new endpoints, change workflows, etc. By forcing operators to upgrade all of their clients to start using API 2.x, it just adds to their burden of upgrading when that time comes. Rather than surgically fixing the clients of the 2% of deprecated API endpoints before upgrading to ATC 5.0, operators would have to fix every single client they have to use API 2.x before upgrading to 5.0. Is that exercise really worth it? Most people would probably be blissfully unaware that the "POST /api/1.1/capabilities" endpoint disappeared (among other deprecated endpoints). Less footguns in the system, and they didn't even have to update their clients. Anyways, circling back to Michael's original question, it sounds like either option we choose, we at least have consensus on this deprecation strategy: - The route IS NOT rewritten from Perl to Golang and a deprecation notice is added to the alert response in the Perl handler If we're content to force all API clients to update their clients to 2.x for zero gain in value, at least we won't be wasting work by rewriting deprecated Perl endpoints to Go. - Rawlin On Fri, Nov 15, 2019 at 7:06 PM Gray, Jonathan <[email protected]> wrote: > > Moving to 2.x means that it's my fault as a consumer of the API if I was > still using the deprecated endpoint (and that change should be part of a > larger upgrade plan). Simply removing the deprecated routes, retroactively, > means I can't rely on having a transition period to get to 2.0 outside of a > large completely down upgrade window. Depending on how the API is used, even > in our own components such as TM, that could prove disastrous. For the older > minor revs like 1.1/1.2, I could live with dropping those as part of a major > project release (including upgrade instructions outlining what releases you > must be on beforehand). Currently 1.3 I view as stable enough to code to or > use a native client library from, and 1.4 is my risk/overhead to take for new > stuff as it's still evolving and improving. Going to 1.5 or 2.0 moves that > forward. The path of least surprise for humans is to expect big shifts on > those major revs. Removal falls under that umbrella and gets as a consumer > not to strictly focus on endpoint->endpoint moves, but rather workflow shifts > and maybe now I have to use the API a different way than before. I believe > getting this right is something that would greatly help us understand does X > component at version 1 work with Y component at version 2, because now you're > asking do components X and Y both speak protocol A. With major releases we > can work through that each time, but it's error prone and requires experts in > all products to agree upon. If you do nothing but consume major releases, > and then conduct your own impact evaluation having read the upgrade > instructions thoroughly, maybe you'd be able to smoothly transition. > > Jonathan G > > > On 11/15/19, 5:24 PM, "Rawlin Peters" <[email protected]> wrote: > > Can someone please explain to me why (A) artificially graduating the > current 1.x API to 2.x, and removing the deprecated endpoints from > 1.x, is better for API clients overall than (B) just removing > deprecated endpoints from 1.x? Existing 1.x clients would gain > absolutely nothing from being forced to update to 2.x like this. > > The way I see the two choices: > A) break every single API client out there > B) break only clients of APIs that we've decided upon as a community > to be deprecated > > IMO, when we have to break things, we should try break the least > amount of clients possible. All I'm looking for is a valuable reason > to choose option A over option B, that isn't just "because SemVer". > > - Rawlin > > On Wed, Nov 13, 2019 at 3:24 PM Schmidt, Andrew (Contractor) > <[email protected]> wrote: > > > > If we decide to get rid of the API versioning it makes sense to me that > we would do that on a major version upgrade, probably to the ATC version. But > I think it would be too early to do it on the 2.0 API version. I like this > compromise +1. I was wondering why we were stuck on the idea that 2.0 had to > be an API rewrite. That seems like it’s a long way out. 2.0 = deprecations > and fixes. 3.0 or 5.0 = api rewrite and maybe version convergence. > > > > On 11/13/19, 11:52 AM, "[email protected]" <[email protected]> > wrote: > > > > Lemme lead with: making breaking changes on a major version boundary > > gets +1 from me - no reason the API needs to be fully redesigned for > > any particular number. > > > > But I just gotta bring this up again: if we want to be able to > > deprecate an endpoint for a major release and remove it in the next > > without needing to deal with the messiness of API versions, we could > > always just get rid of API versions. Major ATC revisions should be > > easier to come by with our new release strategy. > > > > On Wed, 2019-11-13 at 10:23 -0700, Rawlin Peters wrote: > > > > It is a compromise - it's an inconvenience to users, forcing > them > > > > to fix > > > scripts to upgrade. Which isn't good for user experience either; > but > > > it > > > doesn't outright break people. It's a compromise I'm willing to > live > > > with. > > > > > > It will literally outright break clients (*all* of them, not just > > > *some*). We have been down this road before, so I'll just repeat > what > > > I said when this exact conversation came up months ago[1]: > > > > > > "So it basically boils down to: > > > A) promote Go 1.x routes to 2.x and eventually break ALL users of > the > > > 1.x API when the time comes to remove 1.x > > > B) deprecate and remove some 1.x Perl routes and POSSIBLY break > SOME > > > users" > > > > > > Either way we slice and dice it with versioning, removing an > endpoint > > > means that clients of that endpoint will break. By graduating only > > > the > > > "good stuff" from 1.x to 2.x, and removing endpoints from 1.x, we > > > will > > > unnecessarily break all 1.x clients when the time comes to remove > the > > > rest of 1.x. Instead, we can choose to break only clients of 1.x > APIs > > > that we already deemed dangerous, unnecessary, useless, etc. > enough > > > to > > > deprecate. > > > > > > Why break 100% of our API clients when we can choose to break > only a > > > small number of clients that happen to use > > > dangerous/unnecessary/useless APIs? Why would we break our API > > > clients > > > unnecessarily for no added value? There is no sense in blindly > > > adhering to Semantic Versioning just for the sake of adhering to > > > Semantic Versioning. Instead, use it as a guideline, and break the > > > rules where it makes the most sense to do so. > > > > > > - Rawlin > > > > > > [1] > > > > https://urldefense.com/v3/__https://lists.apache.org/thread.html/15c6bfcdce0ac00a3a8612a96c3d668a8a5a16c6ca9401de9473d31c@*3Cdev.trafficcontrol.apache.org*3E__;JSU!rx_L75ITgOQ!V_b3E9iEYqNlZus18f-sVuofwuyxapCthg3LdO4NxyMCU39b7mdf4klP5KNR7XZ7d3Uc$ > > > > > > On Wed, Nov 13, 2019 at 9:43 AM Robert O Butts <[email protected]> > > > wrote: > > > > > But if it's really valuable to remove these endpoints, maybe > it's > > > > appropriate to bump the major version and ask everybody to > update > > > > their > > > > clients? > > > > > > > > +1 > > > > > > > > This sounds like the best compromise to me. It addresses both > sides > > > > - it > > > > lets us remove endpoints, and it also prevents breaking users. > > > > > > > > It is a compromise - it's an inconvenience to users, forcing > them > > > > to fix > > > > scripts to upgrade. Which isn't good for user experience either; > > > > but it > > > > doesn't outright break people. It's a compromise I'm willing to > > > > live with. > > > > > > > > This actually lets us do at least 3 things: > > > > 1. Remove "deprecated" endpoints. > > > > 2. Remove the unmaintained dangerously-large Cache Config > > > > endpoints. > > > > 3. Lets us do Layered Profiles, changing Servers and DSes to > have > > > > arrays of > > > > Profiles instead of a single Profile, without hacky backward- > > > > compatibility > > > > attempts. > > > > > > > > To clarify, `/api/2.0` would be essentially the current API, > just > > > > with the > > > > above breaking changes (possibly others) which require a SemVer > > > > major > > > > version increase. And the big API rewrite/redesign we've been > > > > calling "2.0" > > > > would become `/api/3.0` or some future version. > > > > > > > > It's not perfect, but it seems to address the most concerns on > all > > > > sides. > > > > Thoughts? Objections? > > > > > > > > > > > > On Wed, Nov 13, 2019 at 8:45 AM Chris Lemmons > <[email protected]> > > > > wrote: > > > > > > > > > For the endpoints, I agree in part and disagree in part. > > > > > > > > > > Agree for these two: > > > > > - the endpoints are dangerous and pose a risk to the overall > > > > > Traffic > > > > > Control system > > > > > - the task of rewriting the endpoints from Perl to Go > represents > > > > > an > > > > > unreasonable amount of work for the project > > > > > > > > > > And I would add: > > > > > - the endpoint does not serve its intended purpose and cannot > > > > > reasonably > > > > > be used > > > > > - the endpoint has been made unsafe or unusable by new > features > > > > > added to TC > > > > > > > > > > Disagree for these two: > > > > > - the endpoints have been obsoleted by other endpoints > > > > > - the endpoints don't seem to serve a true purpose or don't > seem > > > > > to be > > > > > valuable to the project anymore > > > > > > > > > > For the most part, obsoleted endpoints can be maintained with > a > > > > > shim > > > > > into the new code. If they would require "an unreasonable > amount > > > > > of > > > > > work", it might make sense to remove them, then. > > > > > > > > > > Things that don't seem valuable usually fall into another > > > > > category > > > > > like "cannot reasonably be used", but if the only reason for > > > > > deprecation is that the feature is going away, we should > really > > > > > support it according to our promise. > > > > > > > > > > But if it's really valuable to remove these endpoints, maybe > it's > > > > > appropriate to bump the major version and ask everybody to > update > > > > > their clients? > > > > > > > > > > On Tue, Nov 12, 2019 at 5:27 PM Rawlin Peters > <[email protected]> > > > > > wrote: > > > > > > The real crux of the issue here is whether or not we agree > as a > > > > > > project that we can move forward with this plan that I > believe > > > > > > to be a > > > > > > good compromise that meets most concerns but "breaks the > > > > > > Semantic > > > > > > Versioning promise": > > > > > > 1) don't rewrite deprecated endpoints from Perl to Go > > > > > > 2) in the Perl handler, add a deprecation notice to the > > > > > > "alerts" > > > > > > section of the response > > > > > > 3) once a deprecated API has been deprecated for one major > > > > > > release, > > > > > > delete it from the API in the subsequent major release > (e.g. if > > > > > > deprecation notices are added and released in ATC 4.x, we > can > > > > > > remove > > > > > > the deprecated endpoint in ATC 5.x) > > > > > > > > > > > > Rob, from your latest proposal to remove the ATS config API > > > > > > endpoints > > > > > > from 1.x, I thought we were past this issue already and > could > > > > > > finally > > > > > > agree to remove deprecated endpoints from 1.x (following > steps > > > > > > 1-3 > > > > > > above) without requiring an all-new 2.x API. Are we not > > > > > > actually past > > > > > > that? You came up with valid reasons for applying these > steps > > > > > > to the > > > > > > ATS config API endpoints, and I think it's fair to say that > we > > > > > > can > > > > > > come up with valid reasons for applying these steps to other > > > > > > deprecated API endpoints as well. > > > > > > > > > > > > It seems like the real problem is actually agreeing on which > > > > > > _specific_ endpoints we should be able to follow steps 1-3 > for. > > > > > > IMO I > > > > > > think it should apply to any 1.x endpoints that we have > valid > > > > > > reasons > > > > > > for not carrying forward to Go, including but not limited > to: > > > > > > - the endpoints are dangerous and pose a risk to the overall > > > > > > Traffic > > > > > > Control system > > > > > > - the endpoints have been obsoleted by other endpoints > > > > > > - the endpoints don't seem to serve a true purpose or don't > > > > > > seem to be > > > > > > valuable to the project anymore > > > > > > - the task of rewriting the endpoints from Perl to Go > > > > > > represents an > > > > > > unreasonable amount of work for the project > > > > > > > > > > > > Obviously most of these reasons are subjective, so we'd > have to > > > > > > come > > > > > > to some level of consensus on deprecating particular > endpoints > > > > > > first, > > > > > > but we should be able to follow steps 1-3 for whatever > > > > > > endpoints we > > > > > > decide. > > > > > > > > > > > > Making breaking changes shouldn't be taken lightly, but we > need > > > > > > a path > > > > > > forward that allows some calculated, breaking changes where > > > > > > necessary. > > > > > > Otherwise, we are unnecessarily holding the project back. > > > > > > > > > > > > - Rawlin > > > > > > > > > > > > On Tue, Nov 12, 2019 at 11:44 AM Jeremy Mitchell < > > > > > > [email protected]> > > > > > wrote: > > > > > > > Yeah, technically a "rewrite" of 1.x means shifting the > > > > > > > implementation > > > > > from > > > > > > > Perl to Go for ALL of 1.x. If you leave a subset of 1.x in > > > > > > > Perl, then > > > > > you > > > > > > > have to call it a "partial rewrite". > > > > > > > > > > > > > > So, assuming we want to do a full rewrite (no more perl > for > > > > > > > the api > > > > > > > whatsoever), I think that means every 1.x api endpoint > (good > > > > > > > or bad) > > > > > needs > > > > > > > to be rewritten to Go so we can remove the dependency on > > > > > > > perl. > > > > > > > > > > > > > > However, I still like marking many as "deprecated" so we > > > > > > > train users > > > > > to use > > > > > > > "better" endpoints that we envision will exist in 2.x and > > > > > > > beyond. > > > > > > > > > > > > > > Jeremy > > > > > > > > > > > > > > On Tue, Nov 12, 2019 at 11:22 AM Hoppal, Michael < > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > We do not have a lot of deprecated routes as of now as > we > > > > > > > > focused on > > > > > > > > routes that are more used first. > > > > > > > > > > > > > > > > I know on a lot of the remaining rewrites there has been > > > > > > > > discussion > > > > > on the > > > > > > > > matching Github issues on potential deprecation (I > cannot > > > > > > > > think of > > > > > the > > > > > > > > exact count but enough to warrant this email __) > > > > > > > > > > > > > > > > It is key to note that I am not arguing for removal of > > > > > > > > endpoints > > > > > within > > > > > > > > the current API version but if there is an endpoint we > > > > > > > > agree should > > > > > be > > > > > > > > deprecated going forward to put a message within the > > > > > > > > response and on > > > > > the > > > > > > > > docs. > > > > > > > > > > > > > > > > And thanks for the input sounds like you are +1 on > > > > > > > > rewriting all > > > > > routes to > > > > > > > > Golang as Brennan is. > > > > > > > > > > > > > > > > On 11/12/19, 9:06 AM, "Robert O Butts" <[email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > Whether we rewrite a route in Go is an > implementation > > > > > > > > detail. To > > > > > the > > > > > > > > interface, to users, it doesn't matter whether a > route > > > > > > > > is > > > > > rewritten or > > > > > > > > not. > > > > > > > > > > > > > > > > But our API follows Semantic Versioning, in order to > > > > > > > > not break > > > > > users. > > > > > > > > We > > > > > > > > can't just remove endpoints that some of us don't > use, > > > > > > > > and > > > > > assume other > > > > > > > > people don't, maybe people who never speak up on the > > > > > > > > mailing > > > > > list. > > > > > > > > We'll > > > > > > > > never gain a big userbase if we keep breaking users. > > > > > > > > > > > > > > > > Per the _project_ SemVer, once we have API 2.0, we > can > > > > > > > > deprecate > > > > > API > > > > > > > > 1.x, > > > > > > > > and in the next major _project_ release, remove API > > > > > > > > 1.x. > > > > > Irrespective > > > > > > > > of > > > > > > > > Perl or Go. > > > > > > > > > > > > > > > > My big concern is, API 2.0 is a big project. How > long > > > > > > > > has the > > > > > rewrite > > > > > > > > to Go > > > > > > > > taken? Do we really believe designing and > implementing > > > > > > > > a > > > > > completely > > > > > > > > new API > > > > > > > > will be any less time? > > > > > > > > > > > > > > > > I don't want killing Perl to have to wait on that. > > > > > > > > > > > > > > > > I know it feels like a waste to rewrite routes that > you > > > > > > > > don't > > > > > use, and > > > > > > > > probably few people do. But that's the cost of a > stable > > > > > > > > project. > > > > > How > > > > > > > > many > > > > > > > > "deprecated" routes are there? If it comes down to > > > > > > > > taking the > > > > > > > > development > > > > > > > > time to rewrite them so we can kill Perl faster, or > > > > > > > > leaving Perl > > > > > > > > around, I > > > > > > > > vote we just do the work and kill Perl. > > > > > > > > > > > > > > > > >If we don't rewrite them, then Perl will last until > > > > > > > > API 2.0 has > > > > > been > > > > > > > > designed, released and then *another full major > release > > > > > > > > cycle*. > > > > > That's > > > > > > > > way > > > > > > > > too long to have two codebases for the same > component, > > > > > > > > IMO, > > > > > especially > > > > > > > > since the rewrite is already 50% complete. > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 12, 2019 at 8:54 AM ocket 8888 < > > > > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > I vote that by and large we DO rewrite them, with > > > > > > > > exceptions > > > > > for > > > > > > > > routes > > > > > > > > > that just plain don't work, even in Perl. Those > are > > > > > > > > few, > > > > > though. > > > > > > > > > > > > > > > > > > If we don't rewrite them, then Perl will last > until > > > > > > > > API 2.0 > > > > > has been > > > > > > > > > designed, released and then *another full major > > > > > > > > release cycle*. > > > > > > > > That's way > > > > > > > > > too long to have two codebases for the same > > > > > > > > component, IMO, > > > > > > > > especially > > > > > > > > > since the rewrite is already 50% complete. > > > > > > > > > > > > > > > > > > On Tue, Nov 12, 2019 at 8:43 AM Hoppal, Michael < > > > > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > As the Traffic Ops API is being rewritten from > Perl > > > > > > > > to Golang > > > > > > > > there has > > > > > > > > > > been several routes that have been deprecated > and > > > > > > > > probably > > > > > more to > > > > > > > > come. > > > > > > > > > > > > > > > > > > > > In the deprecation efforts I have seen two > > > > > > > > strategies: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > * The route IS NOT rewritten from Perl to > > > > > > > > Golang and a > > > > > > > > deprecation > > > > > > > > > > notice is added to the alert response in the > Perl > > > > > > > > handler > > > > > > > > > > * The route IS rewritten from Perl to Golang > > > > > > > > and a > > > > > deprecation > > > > > > > > notice > > > > > > > > > > is added to the alert response in the Golang > > > > > > > > handler > > > > > > > > > > > > > > > > > > > > I think we should have consistency in our > approach > > > > > > > > and > > > > > wanted to > > > > > > > > get > > > > > > > > > > people’s thoughts. > > > > > > > > > > > > > > > > > > > > I would vote that we do not rewrite a deprecated > > > > > > > > route from > > > > > Perl to > > > > > > > > > Golang. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > Michael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
