Hey Nir,
I think part of the motivation for doing this in Traffic Router rather
than the Caching Proxy is separation of concerns. TR is already
concerned with routing a client to the best cache based upon the
client's location, so TR is already well-equipped to make the decision
of how Delivery Services (origins) should be prioritized based upon
the client's location. That way the Caching Proxy (e.g. ATS) doesn't
need to concern itself with its own location, the client's location,
and the location of origins; it just needs to know how to get the
origin's content and cache it. All the client needs to know is that
they have a prioritized list of URLs to choose from; they don't need
to be concerned about origin/edge locations because that
prioritization will be made for them by TR.
The target DSes will have different origins primarily because they
will be in different locations, and the origins should be
interchangeable in terms of the content they provide because a smart
client may fail over to any of the target DSes in a CLIENT_STEERING DS
for the same content.
- Rawlin
On Mon, Mar 12, 2018 at 2:37 PM, Nir Sopher <n...@qwilt.com> wrote:
> Hi Rawlin,
> Can you please add a few word for the motivation behind basing the steering
> target selection on the location of the client?
> As the content goes through the caches, isn't it the job of the cache to
> select the best origin for the cache? Why the client should be the one to
> take the origin location into consideration?
> Why the target DSes have different origins in the first place? Are they
> have different characteristics additionally to their location?
> Thanks,
> Nir
>
> ---------- Forwarded message ----------
> From: Rawlin Peters <rawlin.pet...@gmail.com>
> Date: Mon, Mar 12, 2018 at 9:46 PM
> Subject: Delivery Service Origin Refactor
> To: dev@trafficcontrol.incubator.apache.org
>
>
> Hey folks,
>
> As promised, this email thread will be to discuss how to best
> associate an Origin Latitude/Longitude with a Delivery Service,
> primarily so that steering targets can be ordered/sent to the client
> based upon the location of those targets (i.e. the Origin), a.k.a.
> Steering Target Geo-Ordering. This is potentially going to be a pretty
> large change, so all your feedback/questions/concerns are appreciated.
>
> Here were a handful of bad ideas I had in order to accomplish this DS
> Origin Lat/Long association (feel free to skip to PROPOSED SOLUTION
> below):
>
> 1. Reuse the current MSO (multisite origin) backend (i.e. add the
> origin into the servers table, give it a lat/long from its cachegroup,
> assign the origin server to the DS)
> Pros:
> - reuse of existing db schema, probably wouldn't have to add any new
> tables/columns
> Cons:
> - MSO configuration is already very complex
> - for the simple case of just wanting to give an Origin a lat/long you
> have to create a server (of which only a few fields make sense for an
> Origin), add it to a cachegroup (only name and lat/long make sense,
> won't use parent relationships, isn't really a "group" of origins),
> assign it to a server profile (have to create one first, no parameters
> are needed), and finally assign that Origin server to the delivery
> service (did I miss anything?)
>
> 2. Add Origin lat/long columns to the deliveryservice table
> Pros:
> - probably the most straightforward solution for Steering Target
> Geo-Ordering given that Origin FQDN is currently a DS field.
> Cons:
> - doesn't work well with MSO
> - could be confused with Default Miss Lat/Long
> - if two different delivery services use colocated origins, the same
> lat/long needs entered twice
> - adds yet another column to the crowded deliveryservice table
>
> 3. Add origin lat/long parameters to a Delivery Service Profile
> Pros:
> - Delivery Services using colocated origins could share the same profile
> - no DB schema updates needed
> Cons:
> - profile parameters lack validation
> - still doesn't support lat/long for multiple origins associated with a DS
>
> 4. Add the lat/long to the steering target itself (i.e. where you
> choose weight/order, you'd also enter lat/long)
> Pros:
> - probably the easiest/quickest solution in terms of development
> Cons:
> - only applies lat/long to a steering target
> - using the same target in multiple Steering DSes means having to keep
> the lat/long synced between them all
> - lat/long not easily reused by other areas that may need it in the future
>
>
>
> PROPOSED SOLUTION:
>
> All of those ideas were suboptimal, which is why I think we need to:
> 1. Split Locations out of the cachegroup table into their own table
> with the following columns (cachegroup would have a foreign key to
> Location):
> - name
> - latitude
> - longitude
>
> 2. Split Origins out of the server and deliveryservice tables into
> their own table with the following columns:
> - fqdn
> - protocol (http or https)
> - port (optional, can be inferred from protocol)
> - location (optional FK to Location table)
> - deliveryservice FK (if an Origin can only be associated with a
> single DS. Might need step 3 below for many-to-many)
> - ip_address (optional, necessary to support `use_ip_address` profile
> parameter for using the origin's IP address rather than fqdn in
> parent.config)
> - ip6_address (optional, necessary because we'd have an ip_address
> column for the same reasons)
> - profile (optional, primarily for MSO-specific parameters - rank and
> weight - but I could be convinced that this is unnecessary)
> - cachegroup (optional, necessary to maintain primary/secondary
> relationship between MID_LOC and ORG_LOC cachegroups for MSO)
>
> 3. If many-to-many DSes to Origins will still be possible, create a
> new deliveryservice_origin table to support a many-to-many
> relationship between DSes and origins
> - the rank/weight fields for MSO could be added here possibly, maybe
> other things as well?
>
> 4. Consider constraints in the origin and deliveryservice_origin table
> - must fqdn alone be unique? fqdn, protocol, and port combined?
>
> The process for creating a Delivery Service would change in that
> Origins would have to be created separately and added to the delivery
> service. However, to aid migration to the new way of doing things, our
> UIs could keep the "Origin FQDN" field but the API backend would then
> create a new row in the Origin table and add it to the DS. More
> Origins could then be added (for MSO purposes) to the DS via a new API
> endpoint. MSO configuration would change at least in how Origins are
> assigned to a DS ("server assignments" would then just be for
> EDGE-type servers).
>
> Cachegroup creation also changes in that Locations need to be created
> before associating them to a Cachegroup. However, our UIs could also
> stay the same with the backend API updated to create a Location from
> the Cachegroup request and tie it to the Cachegroup.
>
>
>
> I know there are a lot of backend and frontend implications with these
> changes that would still need to be worked out, but in general does
> this proposal sound good? Questions/concerns/feedback welcome and
> appreciated!
>
> - Rawlin
