I'd propose: * Instead of storing the JSON as blob, use https://www.postgresql.org/doc s/9.2/static/datatype-json.html * Instead of version-then-file request, use a "latest" endpoint with `If-Modified-Since` (https://tools.ietf.org/html/rfc7232#section-3.3). We can also serve each version at endpoints, but `If-Modified-Since` lets us determine whether there's a new snapshot and get it in a single request, both efficiently and using a standard. (We should do the same for the CRConfig).
Also for cache-side config generation, consider https://github.com/apache/incubator-trafficcontrol/pull/151 . It's a prototype and needs work to bring it to production, but the basic functionality is there. Go is safer and faster to develop than Perl, and this is already encapsulated in a library, with both CLI and HTTP microservice examples. I'm certainly willing to help bring it to production. "a single definition file for each CDN which will contain all the information required for any server within that CDN to generate its own configs" Also, long-term, that doesn't scale, nor does the CRConfig. As Traffic Control is deployed with larger and larger CDNs, the CRConfig grows uncontrollably. It's already 5-7mb for us, which takes an approaching-unreasonable amount of time for Traffic Monitor and Router to fetch. This isn't an immediate concern, but long-term, we need to develop a scalable solution, something that says "only give me the data modified since this timestamp". Again, this isn't an immediate crisis. I only mention it now because, if a scalable solution is about the same amount of work, now sounds like a good time. If it's relevantly more work, no worries. But otherwise, +1. We've long needed to Separate our Concerns of Traffic Ops and the cache application. On Mon, Apr 10, 2017 at 5:05 PM, Gelinas, Derek <derek_geli...@comcast.com> wrote: > I would like to propose a new method for ATS config file generation, in > which a single definition file for each CDN which will contain all the > information required for any server within that CDN to generate its own > configs, rather than requesting them from traffic ops. This would be a > version-controlled json file that, when generated, would be stored in a new > table in the traffic ops database as a blob type. This will satisfy > high-availability requirements and allow several versions of the > configuration to be retained for rollback, as well as “freezing” the config > at that moment in time. Combined with cache support coming in 2.1, this > file would only need be generated once per traffic ops server instance. > Instead of queueing servers to update their configurations, the > configuration would be snapshotted similar to the crconfig file and > downloaded by each cache according to their set interval checks – rather > than performing a syncds and checking that the server has been queued for > update, the version number would simply be checked and compared against the > currently active version on the cache itself. Should a difference be found > the server would request the definition file and begin generating > configuration files for itself using the data in the definition file. > > I would like feedback from the community regarding this proposal, and any > suggestions or comments you may have. > > Thanks! > Derek > > Derek Gelinas > IPCDN Engineering > derek_geli...@cable.comcast.com<mailto:derek_geli...@cable.comcast.com> > 603.812.5379 > >
