+1 I was just about to formulate that response. The "dev" list is our discussion forum.
On Tue, Apr 11, 2017 at 9:35 AM, Dave Neuman <[email protected]> wrote: > @Ryan, I think its better to have conversations on the dev list than a wiki > page... > > On Tue, Apr 11, 2017 at 9:01 AM, Durfey, Ryan <[email protected]> > wrote: > > > Started a new wiki page to discuss this here https://cwiki.apache.org/ > > confluence/display/TC/Configuration+Management > > > > I will do my best to summarize the discussion below later today. > > > > Ryan M | 303-524-5099 > > > > > > -----Original Message----- > > From: Eric Friedrich (efriedri) [mailto:[email protected]] > > Sent: Tuesday, April 11, 2017 8:55 AM > > To: [email protected] > > Subject: Re: Proposal for CDN definition file based configuration > > management > > > > A few questions/thoughts, apologies for not in-lining: > > > > 1) If we move away from individually queued updates, we give up the > > ability to make changes and then selectively deploy them. How often do TC > > operations teams make config changes but do not immediately queue > updates. > > (I personally think that we currently have a bit of a tricky situation > > where queuing updates much later can push down an unknowingly large > config > > change to a cache- i.e. many new DS added/removed since last time updates > > were queued maybe months earlier). I wouldn't be sad to see queue updates > > go away, but don't want to cause hardship on operators using that > feature. > > > > 2) If we move away from individually queued updates, how does that affect > > the implicit "config state machine"? Specifically, how will edges know > when > > their parents have been configured and are ready for service? Today we > > don't config an edge cache with a new DS unless the mid is ready to > handle > > traffic as well. > > > > 3) If we move away from individually queued updates, how do we do things > > like unassign a delivery service from a cache? Today we have to snapshot > > CRConfig first to stop redirects to the cache before we queue the update. > > If updates are immediately applied and snapshot is still separate, how do > > we get TR to stop sending traffic to a cache that no longer has the remap > > rule? > > > > 4) Also along the lines of the config state machine, we never really > > closed on if we would make any changes to the queue update/snapshot > > CRConfig flow. If we are looking at redoing how we generate config files, > > it would be great to have consensus on an approach (if not an > > implementation) to remove the need to sequence queue updates and snapshot > > CRConfig. I think the requirement here would be to have Traffic Control > > figure out on its own when to activate/deactivate routing to a cache from > > TR. > > > > 5) I like the suggestion of cache-based config file generation. > > - Caches only retrieve relevant information, so scale proportional to > > number of caches/DSs in the CDN is much better > > - We could modify TR/TM to use the same approach, rather than > > snapshotting a CRConfig. > > - Cache/TR/TM-based config could play a greater role in config state > > machine, rather than having Traffic Ops build static configuration ahead > of > > time. > > > > Downsides > > - Versioning is still possible, but more work than maintaining > snapshots > > of a config file > > - Have to be very careful with API changes, any breakage now impacts > > cache updates. > > > > -Eric > > > > > On Apr 10, 2017, at 9:45 PM, Gelinas, Derek <[email protected] > > > > wrote: > > > > > > Thanks Rob. To your point about scalability: I think that this is more > > scaleable than the current crconfig implementation due to the caching. > > However that is a very valid point and one that has been considered. I've > > started looking into the problem from that angle and hope to have some > more > > solid data soon. I still believe that this is ultimately more scaleable > > than current config implementation, even with the scope caching, but the > > proof will be in the data. > > > > > > Derek > > > > > >> On Apr 10, 2017, at 9:23 PM, Robert Butts <[email protected]> > > wrote: > > >> > > >> I'd propose: > > >> * Instead of storing the JSON as blob, use > > >> https://www.postgresql.org/doc s/9.2/static/datatype-json.html > > >> * Instead of version-then-file request, use a "latest" endpoint with > > >> `If-Modified-Since` > > >> (https://tools.ietf.org/html/rfc7232#section-3.3). We can also serve > > >> each version at endpoints, but `If-Modified-Since` lets us determine > > >> whether there's a new snapshot and get it in a single request, both > > >> efficiently and using a standard. (We should do the same for the > > CRConfig). > > >> > > >> Also for cache-side config generation, consider > > >> https://github.com/apache/incubator-trafficcontrol/pull/151 . It's a > > >> prototype and needs work to bring it to production, but the basic > > >> functionality is there. Go is safer and faster to develop than Perl, > > >> and this is already encapsulated in a library, with both CLI and HTTP > > >> microservice examples. I'm certainly willing to help bring it to > > production. > > >> > > >> > > >> "a single definition file for each CDN which will contain all the > > >> information required for any server within that CDN to generate its > > >> own configs" > > >> > > >> Also, long-term, that doesn't scale, nor does the CRConfig. As > > >> Traffic Control is deployed with larger and larger CDNs, the CRConfig > > >> grows uncontrollably. It's already 5-7mb for us, which takes an > > >> approaching-unreasonable amount of time for Traffic Monitor and > > >> Router to fetch. This isn't an immediate concern, but long-term, we > > >> need to develop a scalable solution, something that says "only give > > >> me the data modified since this timestamp". > > >> > > >> Again, this isn't an immediate crisis. I only mention it now because, > > >> if a scalable solution is about the same amount of work, now sounds > > >> like a good time. If it's relevantly more work, no worries. > > >> > > >> > > >> But otherwise, +1. We've long needed to Separate our Concerns of > > >> Traffic Ops and the cache application. > > >> > > >> > > >> On Mon, Apr 10, 2017 at 5:05 PM, Gelinas, Derek > > >> <[email protected]> > > >> wrote: > > >> > > >>> I would like to propose a new method for ATS config file generation, > > >>> in which a single definition file for each CDN which will contain > > >>> all the information required for any server within that CDN to > > >>> generate its own configs, rather than requesting them from traffic > > >>> ops. This would be a version-controlled json file that, when > > >>> generated, would be stored in a new table in the traffic ops > > >>> database as a blob type. This will satisfy high-availability > > >>> requirements and allow several versions of the configuration to be > > >>> retained for rollback, as well as "freezing" the config at that > > >>> moment in time. Combined with cache support coming in 2.1, this file > > would only need be generated once per traffic ops server instance. > > >>> Instead of queueing servers to update their configurations, the > > >>> configuration would be snapshotted similar to the crconfig file and > > >>> downloaded by each cache according to their set interval checks - > > >>> rather than performing a syncds and checking that the server has > > >>> been queued for update, the version number would simply be checked > > >>> and compared against the currently active version on the cache > > >>> itself. Should a difference be found the server would request the > > >>> definition file and begin generating configuration files for itself > > using the data in the definition file. > > >>> > > >>> I would like feedback from the community regarding this proposal, > > >>> and any suggestions or comments you may have. > > >>> > > >>> Thanks! > > >>> Derek > > >>> > > >>> Derek Gelinas > > >>> IPCDN Engineering > > >>> [email protected]<mailto:[email protected] > > >>> om> > > >>> 603.812.5379 > > >>> > > >>> > > > > > > >
