[
https://issues.apache.org/jira/browse/TC-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15652653#comment-15652653
]
Derek Gelinas commented on TC-32:
---------------------------------
Thus far, I've added api routes for all ATS config files, as well as modified
the manner in which the data is generated. There are three levels - server
specific files, profile specific files, and cdn specific files. Each should
only need to be generated once per file/level.
Profile Specific Level:
50-ats.rules
astats.config - generic_profile
drop_qstring.config
logs_xml.config
plugin.config - generic_profile
storage.config
sysctl.conf - generic_profile
volume.config
Server Level:
12M_facts
ip_allow
parent.config
records.config
remap.config
to_ext_.config
hosting.config
cache.config
CDN Level:
bg_fetch.config
cacheurl.config
hdr_rw_.config
regex_remap_.config
regex_revalidate.config
set_dscp_.config
ssl_multicert.config
url_sig_.config
All files appear to be generating as expected with no errors or missing data.
Next step: clean up and clarify some of the code, change the API file name to
be specific to ATS config file generation, and add options to the routes. The
next stage after that will be to modify the generation process to generate a
list of all files needed per CDN/cachegroup and then generate those files using
the API routes before setting the update flag in the DB.
ORT will also need modification to use these new file locations - we may need
to embed the api routes into the file list in some way.
> Improve efficiency of ATS config generation
> -------------------------------------------
>
> Key: TC-32
> URL: https://issues.apache.org/jira/browse/TC-32
> Project: Traffic Control
> Issue Type: Improvement
> Components: Traffic Ops API
> Reporter: Derek Gelinas
> Assignee: Derek Gelinas
> Fix For: 1.9.0
>
>
> Currently when generating ATS configuration files, each server calls for its
> individual files from traffic ops. This is a very database-intensive process
> that is not scalable enough. I propose the following changes:
> 1) Generate the configuration files only as many times as needed - many files
> are the same (or can be) for the entire CDN or server profile.
> 2) Once generated, the files can be cached locally. Requests for the files
> by ORT will result in pulling down these cached files rather than many
> hundreds of DB queries each time.
> 3) Migrate the routes for these files to the API. Each call will have
> multiple options - a request made with no options would return the cached
> file, another option would return the current DB data, and a third option
> would update the cached file with the current information in the DB.
> 4) When an update is queued, generate and cache the configuration files, then
> activate the update flag for the relevant servers once the cached file
> generation is completed.
> In this way, triggering updates for the caches will result in an initial
> increase in activity as traffic ops generates the files needed, followed by
> the much lower impact of the files themselves being requested by the caches.
> I believe we can cut down the number of server-specific configuration files
> to only 5 intially, and potentially fewer by having ORT fill in certain
> fields on files like records.config with local data during processing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
