Got you. 10x, Nir On Mon, Aug 14, 2017 at 5:37 PM, Dewayne Richardson <dewr...@gmail.com> wrote:
> The goal is to hit all the low hanging fruit first to discover all the > refactor points and infrastructure before we attempt to bite off the bigger > more impactful areas of Traffic Control like Tenancy and Capabilities. > > -Dew > > On Mon, Aug 14, 2017 at 5:22 AM, Nir Sopher <n...@qwilt.com> wrote: > > > +1! > > > > As far as tenancy is concerned, the main logic is held in the > > "Utils::Tenant" class. > > I would be happy to use this class as an entry point to the Golang TO > > development. > > > > BTW, the UT coverage of the tenancy checks is quite extensive. > > Are we going to migrate the UTs as well, write new UTs in Golang, or keep > > working with the Perl UTs? > > > > Nir > > > > On Wed, Aug 9, 2017 at 11:14 PM, Durfey, Ryan <ryan_dur...@comcast.com> > > wrote: > > > > > Great write up Dewayne. Given the length and forward looking nature of > > > the topic I created a wiki page around this under traffic ops. Please > > > continue discussion in the email thread and I will summarize any > changes > > > there. > > > > > > https://cwiki.apache.org/confluence/display/TC/Golang+ > > > Traffic+Ops+Replacement+-+Vampire+Proxy > > > > > > Ryan Durfey M | 303-524-5099 > > > CDN Support (24x7): 866-405-2993 or cdn_supp...@comcast.com<mailto: > > > cdn_supp...@comcast.com> > > > > > > > > > From: Dewayne Richardson <dewr...@gmail.com> > > > Reply-To: "dev@trafficcontrol.incubator.apache.org" < > > > dev@trafficcontrol.incubator.apache.org> > > > Date: Wednesday, August 9, 2017 at 1:03 PM > > > To: "dev@trafficcontrol.incubator.apache.org" < > > > dev@trafficcontrol.incubator.apache.org> > > > Subject: Traffic Ops Golang Rewrite > > > > > > Sorry for the TL;DR, but a lot of information needed to be conveyed. > So, > > > based upon the TO Rewrite discussions Rob Butts and I have been working > > on > > > a Golang proxy (Mark Torluemke, affectionately calls "the Vampire > Proxy") > > > that initially implements the */monitoring* endpoint to lay down the > > > foundation for rewriting more Traffic Ops endpoints in Go. The goal is > > to > > > do a straight rewrite in Go (that implements any old API's as well as > any > > > new ones that follows the /api/1.2 format). The intent is to make this > > > proxy 100% backward compatible (including any HTTP header requirements) > > to > > > keep the existing TO API clients from breaking. > > > > > > This PR is significant because when postinstall runs (after this PR is > > > merged) it switches the ports according to the following: > > > > > > *TO Port Change Overview* > > > Port *443* will be owned by Golang proxy > > > > > > Port *60443* will now be owned by the Mojolicious/Perl Hypnotoad > service > > > > > > See */opt/traffic_ops/app/conf/cdn.conf* for a new property > > > *traffic_ops_golang_port > > > => '443'*. > > > > > > *Important Operational Changes:* > > > > > > *traffic_ops service* > > > The Golang Proxy Service is now combined with the *traffic_ops* > service, > > so > > > when traffic_ops is restarted so is the Golang Proxy Service. Since > the > > > Golang service is a proxy any APIs that are not implemented in the > Golang > > > Service will be forwarded to the existing TO Perl API. Also, If the > API > > is > > > implemented in the Golang Service the response is serviced by the Proxy > > > (where it will access the Postgres database as needed). > > > > > > *traffic_ops logs* > > > *access.log - *old access.log is now renamed to *perl_access.log*, and > > the > > > Go proxy now takes over the *access.log* while logging in the *exact* > > > format > > > as before (no monitoring or tooling changes are required) > > > *traffic_ops_golang.log* - this is a new file where any errors/debug > will > > > be logged from the Go proxy. > > > *perl_access.log - the existing Mojolicious access.log gets a new name* > > > *traffic_ops.log* - existing Mojolicious debug file (no change) > > > > > > There was a lot of debate and discussion about how to move forward and > > this > > > approach was less impactful to operations (which basically means less > > work > > > to move toward Go). Overtime, the goal is to do a rewrite of all of > the > > > relevant endpoints that are in Mojolicious into Go (with a heavy focus > on > > > modularity and unit testing, for a future release with Micro Services). > > > > > > *What about the Qwilt contributions of API Gateway, Capabilities, and > > > Tenancy wasn't that a thing? * > > > Yes, and it still is. As for the API Gateway we will start "absorbing" > > the > > > code that *Amir* Yeshurun, so graciously contributed for the APIGW and > > > JWT. For the API Capabilities that *Naama Shoresh* also generously > > > contributed (all capabilities will be directly accessed from those > > > capability tables), please see the *Roadmap* below. And last but not > > > least, once once we are confident that tenancy works as designed, we > will > > > begin porting all the tenancy hooks as well as any apis related to > > tenancy > > > (tenant apis, user apis, delivery service apis). All of the features > that > > > Qwilt built will be used in some form or fashion. > > > > > > *How do I jump in?* > > > We are going to take a "Biggest Bang for our Buck" approach to the TPv2 > > UI > > > and TO Golang API. The following roadmap is an initial plan of attack > > (of > > > course subject to debate). > > > > > > *Roadmap* > > > - The TO API is READ heavy so all READ endpoints will be rewritten > first > > > (avoiding Tenancy endpoints, like Users and Delivery Services) > > > - Circle back to the CREATE, UPDATE, DELETEs where more Golang > foundation > > > will have to be developed (avoiding Tenancy endpoints) > > > - Implement Capabilities in the new TPv2 to help manage the > > > Roles/Capabilities with ease > > > - Evaluate Capabilities and design the api_capabilities data around > that > > > - Hook in capabilities (once the capabilities design has settled and > > > database is correct in production) > > > - Evaluating the existing Tenancy Rules > > > - Rewrite Tenancy in Golang > > > - Tackle the 'Tenancy' endpoints in Golang, /deliveryservices, /users > > etc. > > > > > > *Reasons to get excited!* > > > > > > - *Improved PR's: *(Dan Kirkwood is helping us automate Jenkins/CI for > > > better PR quality so components like Documentation, Unit Testing are > > first > > > class citizens. The more required items checked off the PR checklist > > makes > > > PR merging MUCH easier. > > > > > > - *Testing:* We have plans in the works to move testing to the > forefront > > > of our development discussions. The hope is a fresh approach to Unit > > > Testing (in a modern Golang world) where Go tools for topics like "Code > > > Coverage" will help us improve our automated testing, and allow us to > > > "Refactor Mercilessly" > > > > > > - *Performance:* My initial testing for the new* /monitoring* Go > endpoint > > > shows an average 20x faster than the Mojolicous Perl equivalent running > > on > > > the same host. More efficiency endpoints mean more "scale" and less > > > production $$. > > > > > > I will work to produce a list of initial endpoints and publish to the > dev > > > list (along with the JIRA Tickets with the requirements to help manage > > the > > > priorities). The following PR (which I plan on merging today, if not > > > tomorrow) has a section that highlights the foundational features that > > the > > > existing TO API had that we had to build in Go, ie: logging, config > > > files, etc. > > > > > > See the high level features implemented in the section *Things left > > before > > > this can be merged:* > > > > > > TC JIRA for this Change: > > > https://issues.apache.org/jira/browse/TC-447 > > > https://github.com/apache/incubator-trafficcontrol/pull/729 > > > > > > Expect more information about our progress and we hope to be as > > transparent > > > as possible, > > > > > > -Dewayne Richardson > > > > > > > > >
