+1 on building the CZF in the database. Jan tried to go down that rabbit hole but realized it was a pretty hard problem to solve. I think this is something we might want to re-visit. Maybe this is something we should discuss at our meetup and then update this thread with our decisions?
On Mon, Mar 12, 2018 at 11:25 AM, Rawlin Peters <[email protected]> wrote: > @VijayAnand: > > Right, a Coverage Zone that doesn't map to a Cache Group in TO won't > be chosen as a backup in case of failure, but you could have a > Coverage-Zone-not-in-TO that configures Coverage-Zones-in-TO as > backups. But, I think the general sentiment right now is that all > Coverage Zones in the CZF should map back to Cache Groups in TO, so > the backup config should also be done via the Cache Group API. > > So from the Traffic Router perspective, the process should become: > 1. Rather than parsing from the CZF into the NetworkNode class, parse > Cache Group backup config from the CRConfig into the existing > CacheLocation class > 2. in the DS request flow, the NetworkNode will map back to a > registered CacheLocation which would contain the backup CG config > > The rest of the PR's behavior should stay the same, it's just a matter > of the config being located in a different class. To give you an idea > of how I would format it in the CRConfig (so you know how to parse it > out), here is a snippet of "edgeLocations" from CRConfig.json: > > "edgeLocations": { > "edge-cg-1": { > "latitude": 1.00, > "longitude": 2.00, > "backupLocations": { > "list": ["edge-cg-2"], > "fallbackToClosest": false > } > }, > "edge-cg-2": { > "latitude": 3.00, > "longitude": 4.00 > }, > } > > The "backupLocations" section would still remain optional (if missing, > follow existing behavior of falling back to next closest CG). Existing > defaults in the current PR should remain the same. > > How would you feel about making those changes in your PR? Feel free to > tackle the new TO API and Traffic Portal changes too if you want, but > I don't want to burden you with this unexpected work if you don't want > it. I (or another willing contributor) could work on the necessary TO > API and Traffic Portal changes sometime in the near future and > integrate them with your Traffic Router enhancement. > > - Rawlin > > > On Mon, Mar 12, 2018 at 7:39 AM, [email protected] > <[email protected]> wrote: > > > > Rawlin, > > > > I believe the following statement is not correct. > > > > <Snip> > > However, after reading your initial proposal below, it sounds like you > > might have Coverage Zones in your CZF that don't necessarily map back > > to Cache Groups in TO. Might that be the case? > > </Snip> > > > > We can have Coverage Zones in CZF which don’t necessarily map in to TO’s > configured list of Cache Groups. But then , it won’t be chosen as a valid > backup in case of failure. > > > > For example: > > GROUP1 and GROUP2 are Cache Groups configured in TO (and hence > cr-config) , where GROUP3 is not in TO. Even though GROUP3 is specified as > a backup for GROUP1, it wont be chosen in case of GROUP1 failure , since > it is not in TO. > > { > > "coverageZones": { > > "GROUP3": { > > "network6": [ > > "1234:567a::\/64", > > "1234:567b::\/64" > > ], > > "network": [ > > "10.197.89.0\/24" > > ] > > }, > > > > "GROUP2": { > > "network6": [ > > "1234:567a::\/64", > > "1234:567b::\/64" > > ], > > "network": [ > > "10.197.69.0\/24" > > ] > > }, > > "GROUP1": { > > "backupZones":{ > > "list": ["GROUP3"],? This wont be chosen as backup Cache Group in > case of failure , since it is not in crconfig. > > "fallbackToClosestGroup":false > > }, > > "network6": [ > > "1234:5677::\/64", > > "1234:5676::\/64" > > ], > > "network": [ > > "10.126.250.0\/24" > > ] > > } > > } > > } > > > > So, i feel, the existing implementation of specifying backupZones > configuratioin in CZF should be fine. > > > > Thanks, > > Vijayanand S > > > > On 2018/03/09 18:31:56, Rawlin Peters <[email protected]> wrote: > >> Hey Eric (and others), > >> > >> I'm resurrecting this thread because the PR [1] implementing this > >> proposed functionality is just about ready to be merged. The full > >> mailing list discussion can be read here [2] if interested. > >> > >> I've discussed this PR a bit more with my colleagues here at Comcast, > >> and while it provides the functionality we need, we think in the > >> long-term this configuration should live in the Cache Group API in > >> Traffic Ops rather than just the Coverage Zone File. > >> > >> However, after reading your initial proposal below, it sounds like you > >> might have Coverage Zones in your CZF that don't necessarily map back > >> to Cache Groups in TO. Might that be the case? That scenario seems to > >> be allowed by Traffic Router but might not necessarily be "supported" > >> given the CZF docs [3] that state: > >> > "The Coverage Zone File (CZF) should contain a cachegroup name to > network prefix mapping in the form:" > >> > >> If we do indeed "support" this scenario, that would mean that having > >> the backupZone config only in TO wouldn't solve all your use cases if > >> your CZF heavily uses Coverage Zones that don't directly map to a > >> Cache Group in TO. > >> > >> If we should officially support this scenario, then maybe we merge the > >> PR [1] as is, then later we can augment the feature so that we can use > >> the Cache Group API to provide the backupZone config as well as in the > >> CZF. If the config was provided in both the API and the CZF, then the > >> API would take precedent. > >> > >> If this scenario should NOT officially be supported, then I think we > >> should update the PR [1] to have Traffic Router parse the config from > >> CRConfig.json rather than the CZF and augment the Cache Group API to > >> support the backupZone config. I think this would be the most ideal > >> solution, but I also don't want to sign up our contributors for extra > >> work that they weren't planning on doing. I'd be happy to help augment > >> this feature on the TO side. > >> > >> What do you all think of this proposal? TO-only or both TO and CZF? > >> > >> - Rawlin > >> > >> [1] https://github.com/apache/incubator-trafficcontrol/pull/1908 > >> [2] https://lists.apache.org/thread.html/b033b3943c22a606370ad3981fa05f > b0e7039161b88bbc035bc49b25@%3Cdev.trafficcontrol.apache.org%3E > >> [3] http://traffic-control-cdn.readthedocs.io/en/latest/ > admin/traffic_ops/using.html#the-coverage-zone-file-and-asn-table > >> > >> On 2016/12/22 19:28:17, Eric Friedrich (efriedri) <[email protected]> > wrote: > >> > The current behavior of cache group selection works as follows > >> > 1) Look for a subnet match in CZF > >> > 2) Use MaxMind/Neustar for GeoLocation based on client IP. Choose > closest cache group. > >> > 3) Use Delivery Service Geo-Miss Lat/Long. Choose closest cache group. > >> > > >> > > >> > For deployments where IP addressing is primarily private (say > RFC-1918 addresses), client IP Geo Location (#2) is not useful. > >> > > >> > > >> > We are considering adding another field to the Coverage Zone File > that configures an ordered list of backup cache groups to try if the > primary cache group does not have any available caches. > >> > > >> > Example: > >> > > >> > "coverageZones": { > >> > "cache-group-01": { > >> > “backupList”: [“cache-group-02”, “cache-group-03”], > >> > "network6": [ > >> > "1234:5678::\/64”, > >> > "1234:5679::\/64"], > >> > "network": [ > >> > "192.168.8.0\/24", > >> > "192.168.9.0\/24”] > >> > } > >> > > >> > This configuration could also be part of the per-cache group > configuration, but that would give less control over which clients > preferred which cache groups. For example, you may have cache groups in LA, > Chicago and NY. If the Chicago Cache group fails, you may want some of the > Chicago clients to go to LA and some to go to NY. If the backup CG > configuration is per-cg, we would not be able to control where clients are > allocated. > >> > > >> > Looking for opinions and comments on the above proposal, this is > still in idea stage. > >> > > >> > Thanks All! > >> > Eric > >> >
