[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies
zrhoffman commented on a change in pull request #4537: URL: https://github.com/apache/trafficcontrol/pull/4537#discussion_r416895350 ## File path: blueprints/flexible-topologies.md ## @@ -250,11 +250,29 @@ response JSON: # `/deliveryservices` endpoints -All relevant Delivery Service APIs will have their JSON request and response objects modified to include a new `topology` field which references the name of the topology it's assigned to: -```JSON +All relevant Delivery Service APIs will have their JSON request and response objects modified to include the following new fields: +- `topology` - the name of the topology it's assigned to +- `firstHeaderRewrite` - the header_rewrite ATS config that will be applied to the *first tier* caches in the Delivery Service's Topology +- `middleHeaderRewrite` - the header_rewrite ATS config that will be applied to all *middle tier* caches in this Delivery Service's Topology +- `lastHeaderRewrite` - the header_rewrite ATS config that will be applied to all *last tier* caches in this Delivery Service's Topology Review comment: Feel free to resolve this convo since your response makes sense and @jhg03a additionally suggested Edge/Parents/LastParent elsewhere. I don't have permission to or I would. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies
zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies URL: https://github.com/apache/trafficcontrol/pull/4537#discussion_r408915906 ## File path: blueprints/flexible-topologies.md ## @@ -250,11 +250,29 @@ response JSON: # `/deliveryservices` endpoints -All relevant Delivery Service APIs will have their JSON request and response objects modified to include a new `topology` field which references the name of the topology it's assigned to: -```JSON +All relevant Delivery Service APIs will have their JSON request and response objects modified to include the following new fields: +- `topology` - the name of the topology it's assigned to +- `firstHeaderRewrite` - the header_rewrite ATS config that will be applied to the *first tier* caches in the Delivery Service's Topology +- `middleHeaderRewrite` - the header_rewrite ATS config that will be applied to all *middle tier* caches in this Delivery Service's Topology +- `lastHeaderRewrite` - the header_rewrite ATS config that will be applied to all *last tier* caches in this Delivery Service's Topology Review comment: IMO `midHeaderRewrite` sounds too close to the existing `midHeaderRewrite` rules that we have today. It would be nice if `firstHeaderRewrite`/`lastHeaderRewrite` were named something that implies their place in the topology, like `leafHeaderRewrite`/`rootNodeHeaderRewrite`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies
zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies URL: https://github.com/apache/trafficcontrol/pull/4537#discussion_r400510194 ## File path: blueprints/flexible-topologies.md ## @@ -0,0 +1,395 @@ + + +# Flexible Topologies + +## Problem Description + +Today, a Traffic Control CDN is limited to 2 tiers -- *EDGE* and *MID* -- with the option to skip the *MID* tier for certain Delivery Service types (e.g. `HTTP_LIVE` and `HTTP_NO_CACHE`). In addition, a CDN is limited to one global parent hierarchy, which is defined via the `parent_cachegroup` and `secondary_parent_cachegroup` fields of cachegroups. Both of these problems limit a CDN's ability to scale with increased demand and changing usage patterns, and providing the ability to add more tiers to a CDN helps it keep up with that growth. A Topology that works well for one set of Delivery Services might not be ideal for a different set of Delivery Services, and a CDN needs the flexibility to provide the best Topology for any given Delivery Service -- with any number of tiers and custom caching hierarchies. + +## Proposed Change + +Traffic Control will provide the ability to define one or more Topologies, and a Topology can have any number of Delivery Services assigned to it. A Topology will be composed of Cachegroups along with their primary/secondary parent relationships to other Cachegroups as defined by the Topology. + +If a Delivery Service is assigned to a Topology, any `deliveryservice_server` assignments it has to `EDGE` caches will be ignored, because it will be assigned to all caches in the Delivery Service's CDN (filtered by server capabilities) that belong to the Topology's cachegroups. Ideally, this feature will obsolete legacy `deliveryservice_server` assignments, since Topologies negate the need to assign Delivery Services to individual `EDGE` caches. Nonetheless, legacy `deliveryservice_server` assignments will be supported alongside Topology-based Delivery Services for some time until all Delivery Services have been migrated to Topologies. + +### Traffic Portal Impact + +Traffic Portal will need new pages for creating and viewing Topologies, and the Delivery Service form will need to be updated to add a new Topology field for assigning a Delivery Service to a Topology. If a Delivery Service is assigned to a Topology, Traffic Portal should prohibit assigning `EDGE` servers to the Delivery Service (`ORIGIN` servers may still need to be assignable for MSO). + +Since Delivery Services will no longer be constrained to one global Topology as they are today, it would be extremely useful to be able to visualize a Delivery Service's Topology like a tree, where each node in the tree is a cachegroup, and the edges between nodes are the primary/secondary parent relationships between them. Clicking on a particular node would show all the servers in that cachegroup that could serve a request for the Delivery Service. This visualization will most likely be different from the Topology form for creating a Topology and does not necessarily need to be provided by Traffic Portal. + +### Traffic Ops Impact + +Traffic Ops will provide the ability to create Topologies, composed of cachegroups and parent relationships, which will be assignable to one or more Delivery Services. + + REST API Impact + +The following is the JSON representation of a `Topology` object: + +```JSON +{ +"name": "foo", +"description": "a foo topology", +"nodes": [ +{ +"cachegroup": "child-cachegroup", +"parents": [1, 2] +}, +{ +"cachegroup": "parent-cachegroup", +"parents": [] +}, +{ +"cachegroup": "secondary-parent-cachegroup", +"parents": [] +} +] +} +``` + +The following table describes the top-level `Topology` object: + +| field | type| optionality | description | +| --- | --- | --- | --- | +| name| string | required| a unique name for identifying this Topology | +| description | string | required| the description of this Topology| +| nodes | array of `node` sub-objects | required| the set of `nodes` in this topology, similar to an *adjacency list* | + +The following table describes the `node` sub-object: + +| field | type | optionality | description | +| -- | - | --- |
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies
zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies URL: https://github.com/apache/trafficcontrol/pull/4537#discussion_r398689829 ## File path: blueprints/flexible-topologies.md ## @@ -0,0 +1,395 @@ + + +# Flexible Topologies + +## Problem Description + +Today, a Traffic Control CDN is limited to 2 tiers -- *EDGE* and *MID* -- with the option to skip the *MID* tier for certain Delivery Service types (e.g. `HTTP_LIVE` and `HTTP_NO_CACHE`). In addition, a CDN is limited to one global parent hierarchy, which is defined via the `parent_cachegroup` and `secondary_parent_cachegroup` fields of cachegroups. Both of these problems limit a CDN's ability to scale with increased demand and changing usage patterns, and providing the ability to add more tiers to a CDN helps it keep up with that growth. A Topology that works well for one set of Delivery Services might not be ideal for a different set of Delivery Services, and a CDN needs the flexibility to provide the best Topology for any given Delivery Service -- with any number of tiers and custom caching hierarchies. + +## Proposed Change + +Traffic Control will provide the ability to define one or more Topologies, and a Topology can have any number of Delivery Services assigned to it. A Topology will be composed of Cachegroups along with their primary/secondary parent relationships to other Cachegroups as defined by the Topology. + +If a Delivery Service is assigned to a Topology, any `deliveryservice_server` assignments it has to `EDGE` caches will be ignored, because it will be assigned to all caches in the Delivery Service's CDN (filtered by server capabilities) that belong to the Topology's cachegroups. Ideally, this feature will obsolete legacy `deliveryservice_server` assignments, since Topologies negate the need to assign Delivery Services to individual `EDGE` caches. Nonetheless, legacy `deliveryservice_server` assignments will be supported alongside Topology-based Delivery Services for some time until all Delivery Services have been migrated to Topologies. + +### Traffic Portal Impact + +Traffic Portal will need new pages for creating and viewing Topologies, and the Delivery Service form will need to be updated to add a new Topology field for assigning a Delivery Service to a Topology. If a Delivery Service is assigned to a Topology, Traffic Portal should prohibit assigning `EDGE` servers to the Delivery Service (`ORIGIN` servers may still need to be assignable for MSO). Review comment: Structure? Hierarchy? Architecture? Scope? Grouping? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies
zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies URL: https://github.com/apache/trafficcontrol/pull/4537#discussion_r398155528 ## File path: blueprints/flexible-topologies.md ## @@ -0,0 +1,395 @@ + + +# Flexible Topologies + +## Problem Description + +Today, a Traffic Control CDN is limited to 2 tiers -- *EDGE* and *MID* -- with the option to skip the *MID* tier for certain Delivery Service types (e.g. `HTTP_LIVE` and `HTTP_NO_CACHE`). In addition, a CDN is limited to one global parent hierarchy, which is defined via the `parent_cachegroup` and `secondary_parent_cachegroup` fields of cachegroups. Both of these problems limit a CDN's ability to scale with increased demand and changing usage patterns, and providing the ability to add more tiers to a CDN helps it keep up with that growth. A Topology that works well for one set of Delivery Services might not be ideal for a different set of Delivery Services, and a CDN needs the flexibility to provide the best Topology for any given Delivery Service -- with any number of tiers and custom caching hierarchies. + +## Proposed Change + +Traffic Control will provide the ability to define one or more Topologies, and a Topology can have any number of Delivery Services assigned to it. A Topology will be composed of Cachegroups along with their primary/secondary parent relationships to other Cachegroups as defined by the Topology. + +If a Delivery Service is assigned to a Topology, any `deliveryservice_server` assignments it has to `EDGE` caches will be ignored, because it will be assigned to all caches in the Delivery Service's CDN (filtered by server capabilities) that belong to the Topology's cachegroups. Ideally, this feature will obsolete legacy `deliveryservice_server` assignments, since Topologies negate the need to assign Delivery Services to individual `EDGE` caches. Nonetheless, legacy `deliveryservice_server` assignments will be supported alongside Topology-based Delivery Services for some time until all Delivery Services have been migrated to Topologies. + +### Traffic Portal Impact + +Traffic Portal will need new pages for creating and viewing Topologies, and the Delivery Service form will need to be updated to add a new Topology field for assigning a Delivery Service to a Topology. If a Delivery Service is assigned to a Topology, Traffic Portal should prohibit assigning `EDGE` servers to the Delivery Service (`ORIGIN` servers may still need to be assignable for MSO). + +Since Delivery Services will no longer be constrained to one global Topology as they are today, it would be extremely useful to be able to visualize a Delivery Service's Topology like a tree, where each node in the tree is a cachegroup, and the edges between nodes are the primary/secondary parent relationships between them. Clicking on a particular node would show all the servers in that cachegroup that could serve a request for the Delivery Service. This visualization will most likely be different from the Topology form for creating a Topology and does not necessarily need to be provided by Traffic Portal. + +### Traffic Ops Impact + +Traffic Ops will provide the ability to create Topologies, composed of cachegroups and parent relationships, which will be assignable to one or more Delivery Services. + + REST API Impact + +The following is the JSON representation of a `Topology` object: + +```JSON +{ +"name": "foo", +"description": "a foo topology", +"nodes": [ +{ +"cachegroup": "child-cachegroup", +"parents": [1, 2] +}, +{ +"cachegroup": "parent-cachegroup", +"parents": [] +}, +{ +"cachegroup": "secondary-parent-cachegroup", +"parents": [] +} +] +} +``` + +The following table describes the top-level `Topology` object: + +| field | type| optionality | description | +| --- | --- | --- | --- | +| name| string | required| a unique name for identifying this Topology | +| description | string | required| the description of this Topology| +| nodes | array of `node` sub-objects | required| the set of `nodes` in this topology, similar to an *adjacency list* | + +The following table describes the `node` sub-object: + +| field | type | optionality | description | +| -- | - | --- |
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies
zrhoffman commented on a change in pull request #4537: Add blueprint for Flexible Topologies URL: https://github.com/apache/trafficcontrol/pull/4537#discussion_r398135831 ## File path: blueprints/flexible-topologies.md ## @@ -0,0 +1,395 @@ + + +# Flexible Topologies + +## Problem Description + +Today, a Traffic Control CDN is limited to 2 tiers -- *EDGE* and *MID* -- with the option to skip the *MID* tier for certain Delivery Service types (e.g. `HTTP_LIVE` and `HTTP_NO_CACHE`). In addition, a CDN is limited to one global parent hierarchy, which is defined via the `parent_cachegroup` and `secondary_parent_cachegroup` fields of cachegroups. Both of these problems limit a CDN's ability to scale with increased demand and changing usage patterns, and providing the ability to add more tiers to a CDN helps it keep up with that growth. A Topology that works well for one set of Delivery Services might not be ideal for a different set of Delivery Services, and a CDN needs the flexibility to provide the best Topology for any given Delivery Service -- with any number of tiers and custom caching hierarchies. + +## Proposed Change + +Traffic Control will provide the ability to define one or more Topologies, and a Topology can have any number of Delivery Services assigned to it. A Topology will be composed of Cachegroups along with their primary/secondary parent relationships to other Cachegroups as defined by the Topology. + +If a Delivery Service is assigned to a Topology, any `deliveryservice_server` assignments it has to `EDGE` caches will be ignored, because it will be assigned to all caches in the Delivery Service's CDN (filtered by server capabilities) that belong to the Topology's cachegroups. Ideally, this feature will obsolete legacy `deliveryservice_server` assignments, since Topologies negate the need to assign Delivery Services to individual `EDGE` caches. Nonetheless, legacy `deliveryservice_server` assignments will be supported alongside Topology-based Delivery Services for some time until all Delivery Services have been migrated to Topologies. + +### Traffic Portal Impact + +Traffic Portal will need new pages for creating and viewing Topologies, and the Delivery Service form will need to be updated to add a new Topology field for assigning a Delivery Service to a Topology. If a Delivery Service is assigned to a Topology, Traffic Portal should prohibit assigning `EDGE` servers to the Delivery Service (`ORIGIN` servers may still need to be assignable for MSO). Review comment: If we add topologies, what will we rename the "Topology" section to in Traffic Portal? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services