This is an automated email from the ASF dual-hosted git repository. rshah pushed a commit to branch 8.0.x in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit bd00f2f4ac53c91bc8dc3bb0de7518e47cc7aa8a Author: Rima Shah <[email protected]> AuthorDate: Mon Dec 4 13:09:35 2023 -0700 Removed anycast functionality from traffic control (#7877) * Removed anycast functionality from traffic control (#62) * removed anycast functionality from traffic control * added changelog for pr * corrected formatting in traffic_monitor/datareq/datareq.go --------- Co-authored-by: serDrem <[email protected]> * Updated changelog --------- Co-authored-by: Dremin, Sergey (Principal SRE, CDN EngOps) <[email protected]> Co-authored-by: serDrem <[email protected]> (cherry picked from commit 3710addef3a1f42c56461d8f46cc81dfe97049ac) --- CHANGELOG.md | 1 - docs/source/admin/traffic_monitor.rst | 1 - docs/source/glossary.rst | 5 -- docs/source/overview/traffic_monitor.rst | 4 - lib/go-rfc/http.go | 1 - lib/go-tc/crstates.go | 1 - traffic_monitor/cache/astats.go | 18 ----- traffic_monitor/cache/astats_csv.go | 3 - traffic_monitor/cache/cache.go | 8 -- traffic_monitor/cache/data.go | 4 - traffic_monitor/cache/stats_over_http.go | 13 --- traffic_monitor/datareq/crstate.go | 77 +++--------------- traffic_monitor/datareq/crstate_test.go | 135 ------------------------------- traffic_monitor/datareq/datareq.go | 17 +--- traffic_monitor/health/cache.go | 32 ++------ traffic_monitor/manager/statecombiner.go | 2 +- traffic_monitor/todata/todata.go | 56 ------------- traffic_monitor/todata/todata_test.go | 68 ---------------- 18 files changed, 21 insertions(+), 425 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 17a9b1f9f2..fcb5b74085 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,7 +16,6 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). - [#7672](https://github.com/apache/trafficcontrol/pull/7672) *Traffic Control Health Client*: Added peer monitor flag while using `strategies.yaml`. - [#7609](https://github.com/apache/trafficcontrol/pull/7609) *Traffic Portal*: Added Scope Query Param to SSO login. - [#7450](https://github.com/apache/trafficcontrol/pull/7450) *Traffic Ops*: Removed hypnotoad section and added listen field to traffic_ops_golang section in order to simplify cdn config. -- [#7290](https://github.com/apache/trafficcontrol/pull/7302) *Traffic Monitor*: Update TM results with hostname from via header, syncronize health on caches with same service address. - [#7291](https://github.com/apache/trafficcontrol/pull/7291) *Traffic Ops*: Extended Layered Profile feature to aggregate parameters for all server profiles. - [#7314](https://github.com/apache/trafficcontrol/pull/7314) *Traffic Portal*: Added capability feature to Delivery Service Form (HTTP, DNS). - [#7295](https://github.com/apache/trafficcontrol/pull/7295) *Traffic Portal*: Added description and priority order for Layered Profile on server form. diff --git a/docs/source/admin/traffic_monitor.rst b/docs/source/admin/traffic_monitor.rst index fdb7a851cb..cd370c35e9 100644 --- a/docs/source/admin/traffic_monitor.rst +++ b/docs/source/admin/traffic_monitor.rst @@ -79,7 +79,6 @@ traffic_monitor.cfg - ``health.polling.interval`` - ``peers.polling.interval`` - ``heartbeat.polling.interval`` -- ``tm.sameipservers.enabled`` - When set to true, performs an AND operation on the availability statuses of servers with same ip. Any unavailable server(s) with same ip as other server(s) will cause the other server(s) to be set to unavailable. Upon receiving this configuration, Traffic Monitor begins polling :term:`cache server` s. Once every :term:`cache server` has been polled, :ref:`health-proto` state is available via RESTful JSON endpoints and a web browser UI. diff --git a/docs/source/glossary.rst b/docs/source/glossary.rst index 98fcc8c351..e074341b58 100644 --- a/docs/source/glossary.rst +++ b/docs/source/glossary.rst @@ -41,11 +41,6 @@ Glossary - :term:`forward proxy`: Used by Traffic Control for Mid-tier :dfn:`cache servers`. - transparent proxy: These are not used by Traffic Control. If you are interested you can learn more about transparent proxies on `wikipedia <http://en.wikipedia.org/wiki/Proxy_server#Transparent_proxy>`_. - anycast group - Anycast group - Anycast Group - A group of caching HTTP proxy servers that have the same service address that is equal cost multi path routed at the last hop - Cache Group Cache Groups A group of caching HTTP proxy servers that together create a combined larger cache using consistent hashing. Traffic Router treats all servers in a :dfn:`Cache Group` as though they are in the same geographic location, though they are in fact only in the same general area. A :dfn:`Cache Group` has one single set of geographical coordinates even if the :term:`cache servers` that make up the :dfn:`Cache Group` are actually in :term:`Physical Locations`. The :term:`cache servers` in a :d [...] diff --git a/docs/source/overview/traffic_monitor.rst b/docs/source/overview/traffic_monitor.rst index aaff86bf25..48de9a0c47 100644 --- a/docs/source/overview/traffic_monitor.rst +++ b/docs/source/overview/traffic_monitor.rst @@ -55,7 +55,3 @@ The optimistic quorum prevents invalid state propagation caused by a Traffic Mon Protocol Engagement ------------------- Short polling intervals of both the :term:`cache servers` and Traffic Monitor peers help to reduce customer impact of outages. It is not uncommon for a :term:`cache server` to be marked unavailable by Traffic Monitor - in fact, it is business as usual for many CDNs. Should a widely requested video asset cause a single :term:`cache server` to get close to its interface capacity, the Health Protocol will "kick in," and Traffic Monitor marks the :term:`cache server` as unavailable. New clie [...] - -Handling Cache Servers with Same Service IP -------------------------------------------- -Traffic Monitor is able to direct Traffic Router to route traffic to groups of :term:`cache servers` with the same service IP address, but not to an individual :term:`cache server` within that group. To monitor :term:`cache servers` within that group, HTTP :mailheader:`Keep-Alive` header is omitted with the ``health.polling.keepalive`` and ``stat.polling.keepalive`` Traffic Monitor :term:`profile` :term:`parameters` so that the Traffic Monitor source port changes for those connections an [...] \ No newline at end of file diff --git a/lib/go-rfc/http.go b/lib/go-rfc/http.go index d8e3421a07..1a538d7b5c 100644 --- a/lib/go-rfc/http.go +++ b/lib/go-rfc/http.go @@ -43,7 +43,6 @@ const ( Age = "Age" // RFC7234§5.1 Location = "Location" // RFC7231§7.1.2 Authorization = "Authorization" // RFC7235§4.2 - Via = "Via" // RFC3261§8.1.1.7 Cookie = "Cookie" // RFC7873 ) diff --git a/lib/go-tc/crstates.go b/lib/go-tc/crstates.go index 4154b9f363..a7d99bffbc 100644 --- a/lib/go-tc/crstates.go +++ b/lib/go-tc/crstates.go @@ -44,7 +44,6 @@ type IsAvailable struct { DirectlyPolled bool `json:"-"` Status string `json:"status"` LastPoll time.Time `json:"lastPoll"` - LastPollV6 time.Time `json:"lastPollV6"` } // NewCRStates creates a new CR states object, initializing pointer members. diff --git a/traffic_monitor/cache/astats.go b/traffic_monitor/cache/astats.go index 48c234ec86..00236e3f7a 100644 --- a/traffic_monitor/cache/astats.go +++ b/traffic_monitor/cache/astats.go @@ -33,21 +33,17 @@ import ( "errors" "fmt" "io" - "regexp" "strings" "github.com/apache/trafficcontrol/v8/lib/go-log" - "github.com/apache/trafficcontrol/v8/lib/go-rfc" "github.com/apache/trafficcontrol/v8/traffic_monitor/dsdata" "github.com/apache/trafficcontrol/v8/traffic_monitor/poller" "github.com/apache/trafficcontrol/v8/traffic_monitor/todata" - jsoniter "github.com/json-iterator/go" ) func init() { registerDecoder("astats", astatsParse, astatsPrecompute) - hostnameRegex = regexp.MustCompile(`(?:http|https)/\d+\.\d+ ([A-Za-z0-9\-]{0,61})`) } // AstatsSystem represents fixed system stats returned from the @@ -120,22 +116,8 @@ func astatsParse(cacheName string, rdr io.Reader, pollCTX interface{}) (Statisti astats.Ats["system.proc.loadavg"] = astats.System.ProcLoadavg astats.Ats["system.proc.net.dev"] = astats.System.ProcNetDev - via := ctx.HTTPHeader.Get(rfc.Via) - if via != "" { - viaRegexSubmatch := hostnameRegex.FindStringSubmatch(via) - if len(viaRegexSubmatch) > 0 { - astats.Ats[rfc.Via] = viaRegexSubmatch[1] - } - } return stats, astats.Ats, nil } else if ctype == "text/csv" { - via := ctx.HTTPHeader.Get(rfc.Via) - if via != "" { - viaRegexSubmatch := hostnameRegex.FindStringSubmatch(via) - if len(viaRegexSubmatch) > 0 { - cacheName = viaRegexSubmatch[1] - } - } return astatsCsvParseCsv(cacheName, rdr) } else { return stats, nil, fmt.Errorf("stats Content-Type (%s) can not be parsed by astats", ctype) diff --git a/traffic_monitor/cache/astats_csv.go b/traffic_monitor/cache/astats_csv.go index 49b0e307de..94d3e5b624 100644 --- a/traffic_monitor/cache/astats_csv.go +++ b/traffic_monitor/cache/astats_csv.go @@ -28,7 +28,6 @@ import ( "strings" "github.com/apache/trafficcontrol/v8/lib/go-log" - "github.com/apache/trafficcontrol/v8/lib/go-rfc" ) type astatsDataCsv struct { @@ -129,7 +128,5 @@ func astatsCsvParseCsv(cacheName string, data io.Reader) (Statistics, map[string return stats, nil, fmt.Errorf("cache '%s' had no interfaces", cacheName) } - statMap[rfc.Via] = cacheName - return stats, statMap, nil } diff --git a/traffic_monitor/cache/cache.go b/traffic_monitor/cache/cache.go index cb2002ea0b..5a0442e83e 100644 --- a/traffic_monitor/cache/cache.go +++ b/traffic_monitor/cache/cache.go @@ -20,13 +20,10 @@ package cache */ import ( - "fmt" "io" - "regexp" "time" "github.com/apache/trafficcontrol/v8/lib/go-log" - "github.com/apache/trafficcontrol/v8/lib/go-rfc" "github.com/apache/trafficcontrol/v8/lib/go-tc" "github.com/apache/trafficcontrol/v8/traffic_monitor/todata" ) @@ -37,8 +34,6 @@ type Handler struct { ToData *todata.TODataThreadsafe } -var hostnameRegex *regexp.Regexp - func (h Handler) ResultChan() <-chan Result { return h.resultChan } @@ -316,9 +311,6 @@ func (handler Handler) Handle(id string, rdr io.Reader, format string, reqTime t } result.Time = time.UnixMilli(int64(valInt)) } - if value, ok := miscStats[rfc.Via]; ok { - result.ID = fmt.Sprintf("%v", value) - } result.Statistics = stats result.Miscellaneous = miscStats diff --git a/traffic_monitor/cache/data.go b/traffic_monitor/cache/data.go index bb7f1357b2..c3c19e9aae 100644 --- a/traffic_monitor/cache/data.go +++ b/traffic_monitor/cache/data.go @@ -64,10 +64,6 @@ type AvailableStatus struct { UnavailableStat string // Poller is the name of the poller which set this availability status. Poller string - // Time of last poll - LastPoll time.Time - // Time of v6 last poll - LastPollV6 time.Time } // CacheAvailableStatuses is the available status of each cache. diff --git a/traffic_monitor/cache/stats_over_http.go b/traffic_monitor/cache/stats_over_http.go index 77cc9a6e95..b0c21066f7 100644 --- a/traffic_monitor/cache/stats_over_http.go +++ b/traffic_monitor/cache/stats_over_http.go @@ -25,12 +25,10 @@ import ( "fmt" "io" "math" - "regexp" "strconv" "strings" "github.com/apache/trafficcontrol/v8/lib/go-log" - "github.com/apache/trafficcontrol/v8/lib/go-rfc" "github.com/apache/trafficcontrol/v8/traffic_monitor/poller" "github.com/apache/trafficcontrol/v8/traffic_monitor/todata" @@ -55,7 +53,6 @@ const LOADAVG_SHIFT = 65536 func init() { registerDecoder("stats_over_http", statsOverHTTPParse, statsOverHTTPPrecompute) - hostnameRegex = regexp.MustCompile(`(?:http|https)/\d+\.\d+ ([A-Za-z0-9\-]{0,61})`) } type stats_over_httpData struct { @@ -74,14 +71,6 @@ func statsOverHTTPParse(cacheName string, data io.Reader, pollCTX interface{}) ( ctx := pollCTX.(*poller.HTTPPollCtx) - via := ctx.HTTPHeader.Get(rfc.Via) - if via != "" { - viaRegexSubmatch := hostnameRegex.FindStringSubmatch(via) - if len(viaRegexSubmatch) > 0 { - cacheName = viaRegexSubmatch[1] - } - } - ctype := ctx.HTTPHeader.Get("Content-Type") if ctype == "text/json" || ctype == "text/javascript" || ctype == "application/json" || ctype == "" { @@ -105,8 +94,6 @@ func statsOverHTTPParse(cacheName string, data io.Reader, pollCTX interface{}) ( statMap := sohData.Global - statMap[rfc.Via] = cacheName - if stats.Loadavg, err = parseLoadAvg(statMap); err != nil { return stats, nil, fmt.Errorf("Error parsing loadavg for cache '%s': %v", cacheName, err) } diff --git a/traffic_monitor/datareq/crstate.go b/traffic_monitor/datareq/crstate.go index 92817f377c..54ce4b1643 100644 --- a/traffic_monitor/datareq/crstate.go +++ b/traffic_monitor/datareq/crstate.go @@ -23,14 +23,10 @@ import ( "fmt" "net/http" "net/url" - "strings" "github.com/apache/trafficcontrol/v8/lib/go-log" "github.com/apache/trafficcontrol/v8/lib/go-tc" - "github.com/apache/trafficcontrol/v8/traffic_monitor/health" "github.com/apache/trafficcontrol/v8/traffic_monitor/peer" - "github.com/apache/trafficcontrol/v8/traffic_monitor/threadsafe" - "github.com/apache/trafficcontrol/v8/traffic_monitor/todata" ) func srvTRState( @@ -39,13 +35,11 @@ func srvTRState( combinedStates peer.CRStatesThreadsafe, peerStates peer.CRStatesPeersThreadsafe, distributedPollingEnabled bool, - toData todata.TODataThreadsafe, - monitorConfig threadsafe.TrafficMonitorConfigMap, ) ([]byte, int, error) { _, raw := params["raw"] // peer polling case _, local := params["local"] // distributed peer polling case if raw { - data, err := srvTRStateData(localStates, distributedPollingEnabled, toData, monitorConfig) + data, err := srvTRStateSelf(localStates, distributedPollingEnabled) return data, http.StatusOK, err } @@ -64,11 +58,19 @@ func srvTRState( } } - data, err := srvTRStateData(combinedStates, local && distributedPollingEnabled, toData, monitorConfig) + data, err := srvTRStateDerived(combinedStates, local && distributedPollingEnabled) return data, http.StatusOK, err } +func srvTRStateDerived(combinedStates peer.CRStatesThreadsafe, directlyPolledOnly bool) ([]byte, error) { + if !directlyPolledOnly { + return tc.CRStatesMarshall(combinedStates.Get()) + } + unfiltered := combinedStates.Get() + return tc.CRStatesMarshall(filterDirectlyPolledCaches(unfiltered)) +} + func filterDirectlyPolledCaches(crstates tc.CRStates) tc.CRStates { filtered := tc.CRStates{ Caches: make(map[tc.CacheName]tc.IsAvailable), @@ -82,63 +84,10 @@ func filterDirectlyPolledCaches(crstates tc.CRStates) tc.CRStates { return filtered } -func srvTRStateData(localStates peer.CRStatesThreadsafe, directlyPolledOnly bool, toData todata.TODataThreadsafe, monitorConfig threadsafe.TrafficMonitorConfigMap) ([]byte, error) { - if val, ok := monitorConfig.Get().Config["tm.sameipservers.enabled"]; ok && val.(string) == "true" { - localStatesC := updateStatusSameIpServers(localStates, toData) - if !directlyPolledOnly { - return tc.CRStatesMarshall(localStatesC) - } - return tc.CRStatesMarshall(filterDirectlyPolledCaches(localStatesC)) - } +func srvTRStateSelf(localStates peer.CRStatesThreadsafe, directlyPolledOnly bool) ([]byte, error) { if !directlyPolledOnly { return tc.CRStatesMarshall(localStates.Get()) } - return tc.CRStatesMarshall(filterDirectlyPolledCaches(localStates.Get())) -} - -func updateStatusSameIpServers(localStates peer.CRStatesThreadsafe, toData todata.TODataThreadsafe) tc.CRStates { - localStatesC := localStates.Get() - toDataC := toData.Get() - - for cache, _ := range localStatesC.Caches { - if _, ok := toDataC.SameIpServers[cache]; ok { - // all servers with same ip must be available if they are in reported state - allAvailableV4 := true - allAvailableV6 := true - allIsAvailable := true - for partner, _ := range toDataC.SameIpServers[cache] { - if partnerState, ok := localStatesC.Caches[partner]; ok { - // a partner host is reported but is marked down for exceeding a threshold - // this host also needs to be marked down to divert all traffic for their - // common ip - if strings.Contains(partnerState.Status, string(tc.CacheStatusReported)) && - strings.Contains(partnerState.Status, health.TooHigh.String()) { - if !partnerState.Ipv4Available { - allAvailableV4 = false - } - if !partnerState.Ipv6Available { - allAvailableV6 = false - } - if !partnerState.IsAvailable { - allIsAvailable = false - } - if !allAvailableV4 && !allAvailableV6 && !allIsAvailable { - break - } - } - } - } - newIsAvailable := tc.IsAvailable{} - newIsAvailable.DirectlyPolled = localStatesC.Caches[cache].DirectlyPolled - newIsAvailable.Status = localStatesC.Caches[cache].Status - newIsAvailable.LastPoll = localStatesC.Caches[cache].LastPoll - newIsAvailable.LastPollV6 = localStatesC.Caches[cache].LastPollV6 - newIsAvailable.IsAvailable = localStatesC.Caches[cache].IsAvailable && allIsAvailable - newIsAvailable.Ipv4Available = localStatesC.Caches[cache].Ipv4Available && allAvailableV4 - newIsAvailable.Ipv6Available = localStatesC.Caches[cache].Ipv6Available && allAvailableV6 - - localStatesC.Caches[cache] = newIsAvailable - } - } - return localStatesC + unfiltered := localStates.Get() + return tc.CRStatesMarshall(filterDirectlyPolledCaches(unfiltered)) } diff --git a/traffic_monitor/datareq/crstate_test.go b/traffic_monitor/datareq/crstate_test.go deleted file mode 100644 index d152f0f2cf..0000000000 --- a/traffic_monitor/datareq/crstate_test.go +++ /dev/null @@ -1,135 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ - -package datareq - -import ( - "github.com/apache/trafficcontrol/v8/lib/go-tc" - "github.com/apache/trafficcontrol/v8/traffic_monitor/peer" - "github.com/apache/trafficcontrol/v8/traffic_monitor/todata" - "github.com/apache/trafficcontrol/v8/traffic_ops/traffic_ops_golang/test" - "testing" -) - -func setMockTOData(tod *todata.TOData) { - numCaches := 100 - numDSes := 100 - numCacheDSes := numDSes / 3 - numCGs := 20 - - types := []tc.CacheType{tc.CacheTypeEdge, tc.CacheTypeEdge, tc.CacheTypeEdge, tc.CacheTypeEdge, tc.CacheTypeEdge, tc.CacheTypeMid} - - caches := []tc.CacheName{} - for i := 0; i < numCaches; i++ { - caches = append(caches, tc.CacheName(test.RandStr())) - } - - dses := []tc.DeliveryServiceName{} - for i := 0; i < numDSes; i++ { - dses = append(dses, tc.DeliveryServiceName(test.RandStr())) - } - - cgs := []tc.CacheGroupName{} - for i := 0; i < numCGs; i++ { - cgs = append(cgs, tc.CacheGroupName(test.RandStr())) - } - - serverDSes := map[tc.CacheName][]tc.DeliveryServiceName{} - for _, ca := range caches { - for i := 0; i < numCacheDSes; i++ { - serverDSes[ca] = append(serverDSes[ca], dses[test.RandIntn(len(dses))]) - } - } - - dsServers := map[tc.DeliveryServiceName][]tc.CacheName{} - for server, dses := range serverDSes { - for _, ds := range dses { - dsServers[ds] = append(dsServers[ds], server) - } - } - - serverCGs := map[tc.CacheName]tc.CacheGroupName{} - for _, cache := range caches { - serverCGs[cache] = cgs[test.RandIntn(len(cgs))] - } - - serverTypes := map[tc.CacheName]tc.CacheType{} - for _, cache := range caches { - serverTypes[cache] = types[test.RandIntn(len(types))] - } - - tod.DeliveryServiceServers = dsServers - tod.ServerDeliveryServices = serverDSes - tod.ServerTypes = serverTypes - tod.ServerCachegroups = serverCGs -} - -func TestUpdateStatusSameIpServers(t *testing.T) { - toDataTS := todata.NewThreadsafe() - toData := todata.New() - setMockTOData(toData) - - toData.SameIpServers = map[tc.CacheName]map[tc.CacheName]bool{} - toData.SameIpServers["server1_ip1_up"] = map[tc.CacheName]bool{} - toData.SameIpServers["server1_ip1_up"]["server2_ip1_down"] = true - toData.SameIpServers["server2_ip1_down"] = map[tc.CacheName]bool{} - toData.SameIpServers["server2_ip1_down"]["server1_ip1_up"] = true - - toData.SameIpServers["server3_ip3_up"] = map[tc.CacheName]bool{} - toData.SameIpServers["server3_ip3_up"]["server4_ip3_up"] = true - toData.SameIpServers["server4_ip3_up"] = map[tc.CacheName]bool{} - toData.SameIpServers["server4_ip3_up"]["server3_ip3_up"] = true - - localStates := peer.NewCRStatesThreadsafe() - localStates.AddCache("server1_ip1_up", tc.IsAvailable{IsAvailable: true, Ipv4Available: true, Ipv6Available: true, Status: string(tc.CacheStatusReported)}) - localStates.AddCache("server2_ip1_down", tc.IsAvailable{IsAvailable: false, Ipv4Available: false, Ipv6Available: false, Status: string(tc.CacheStatusReported) + "too high"}) - localStates.AddCache("server3_ip3_up", tc.IsAvailable{IsAvailable: true, Ipv4Available: true, Ipv6Available: true, Status: string(tc.CacheStatusReported)}) - localStates.AddCache("server4_ip3_up", tc.IsAvailable{IsAvailable: true, Ipv4Available: true, Ipv6Available: true, Status: string(tc.CacheStatusReported)}) - localStates.AddCache("server5_ip5_up", tc.IsAvailable{IsAvailable: true, Ipv4Available: true, Ipv6Available: true, Status: string(tc.CacheStatusReported)}) - - toDataTS.SetForTest(*toData) - - localStatesC := updateStatusSameIpServers(localStates, toDataTS) - - if localStatesC.Caches["server1_ip1_up"].IsAvailable == true || - localStatesC.Caches["server1_ip1_up"].Ipv4Available == true || - localStatesC.Caches["server1_ip1_up"].Ipv6Available == true { - t.Error("expected server1_ip1_up to be false for IsAvailable Ipv4Available Ipv6Available") - } - if localStatesC.Caches["server2_ip1_down"].IsAvailable != false || - localStatesC.Caches["server2_ip1_down"].Ipv4Available != false || - localStatesC.Caches["server2_ip1_down"].Ipv6Available != false { - t.Error("expected server2_ip1_up to be false for IsAvailable Ipv4Available Ipv6Available") - } - if localStatesC.Caches["server3_ip3_up"].IsAvailable != true || - localStatesC.Caches["server3_ip3_up"].Ipv4Available != true || - localStatesC.Caches["server3_ip3_up"].Ipv6Available != true { - t.Error("expected server3_ip3_up to be true for IsAvailable Ipv4Available Ipv6Available") - } - if localStatesC.Caches["server4_ip3_up"].IsAvailable != true || - localStatesC.Caches["server4_ip3_up"].Ipv4Available != true || - localStatesC.Caches["server4_ip3_up"].Ipv6Available != true { - t.Error("expected server4_ip3_up to be true for IsAvailable Ipv4Available Ipv6Available") - } - if localStatesC.Caches["server5_ip5_up"].IsAvailable != true || - localStatesC.Caches["server5_ip5_up"].Ipv4Available != true || - localStatesC.Caches["server5_ip5_up"].Ipv6Available != true { - t.Error("expected server5_ip5_up to be true for IsAvailable Ipv4Available Ipv6Available") - } -} diff --git a/traffic_monitor/datareq/datareq.go b/traffic_monitor/datareq/datareq.go index 792e01d805..08d94036df 100644 --- a/traffic_monitor/datareq/datareq.go +++ b/traffic_monitor/datareq/datareq.go @@ -79,27 +79,12 @@ func MakeDispatchMap( } } - // if any server has partners listed in toData.SameIpServers, this is an anycast enabled server - // tm can not control traffic to individual anycast enabled servers, since all of them have the same - // service IP - // It can poll all of them, but will only detect not available status on the last server that becomes - // unavailable. Therefore if any of the servers in the anycast group becomes unavailable, that means - // that the TM can not get to ANY of the anycast group servers and all of them must be marked unavailable - // TM will detect server load and bandwidth limits on all the servers that are responding to polls, and - // are serving traffic - // If TM detects server load issues, it can only control that by diverting traffic to all the servers in the - // anycast group - // If TM detects server bandwidth limits, it must divert traffic to all the servers that are responding and not - // responding in the anycast server group. The ones that are responding are in ECMP, and have the same bandwidth, - // so bandwidth to all must be decreased. The ones that are not responding, can appear to be in reported state, - // and must be marked unavailable to control traffic to the remaining servers - dispatchMap := map[string]http.HandlerFunc{ "/publish/CrConfig": wrap(WrapAgeErr(errorCount, func() ([]byte, time.Time, error) { return srvTRConfig(opsConfig, toSession) }, rfc.ApplicationJSON)), "/publish/CrStates": wrap(WrapParams(func(params url.Values, path string) ([]byte, int) { - bytes, statusCode, err := srvTRState(params, localStates, combinedStates, peerStates, distributedPollingEnabled, toData, monitorConfig) + bytes, statusCode, err := srvTRState(params, localStates, combinedStates, peerStates, distributedPollingEnabled) return WrapErrStatusCode(errorCount, path, bytes, statusCode, err) }, rfc.ApplicationJSON)), "/publish/CacheStatsNew": wrap(WrapParams(func(params url.Values, path string) ([]byte, int) { diff --git a/traffic_monitor/health/cache.go b/traffic_monitor/health/cache.go index 9146585feb..8c7bc0148e 100644 --- a/traffic_monitor/health/cache.go +++ b/traffic_monitor/health/cache.go @@ -46,18 +46,6 @@ const AvailableStr = "available" // available to serve traffic. const UnavailableStr = "unavailable" -type Threshold string - -const ( - NotEqual Threshold = "not equal" - TooLow Threshold = "too low" - TooHigh Threshold = "too high" -) - -func (t Threshold) String() string { - return string(t) -} - // GetVitals Gets the vitals to decide health on in the right format func GetVitals(newResult *cache.Result, prevResult *cache.Result, mc *tc.TrafficMonitorConfigMap) { if newResult.Error != nil { @@ -368,21 +356,13 @@ func CalcAvailability( availStatus.UnavailableStat = aggUnavailableStat } - if result.UsingIPv4 { - availStatus.LastPoll = result.Time - availStatus.LastPollV6 = lastStatus.LastPollV6 - } else { - availStatus.LastPoll = lastStatus.LastPoll - availStatus.LastPollV6 = result.Time - } localStates.SetCache(tc.CacheName(result.ID), tc.IsAvailable{ IsAvailable: availStatus.ProcessedAvailable, Ipv4Available: availStatus.Available.IPv4, Ipv6Available: availStatus.Available.IPv6, DirectlyPolled: true, // we know this cache was directly polled because otherwise we wouldn't have a cache.Result for it Status: availStatus.Why, - LastPoll: availStatus.LastPoll, - LastPollV6: availStatus.LastPollV6, + LastPoll: result.Time, }) if available, ok := localStates.GetCache(tc.CacheName(result.ID)); !ok || available.IsAvailable != lastStatus.ProcessedAvailable { @@ -416,15 +396,15 @@ func CalcAvailability( func exceedsThresholdMsg(stat string, threshold tc.HealthThreshold, val float64) string { switch threshold.Comparator { case "=": - return fmt.Sprintf("%s %s (%.2f == %.2f)", stat, NotEqual, val, threshold.Val) + return fmt.Sprintf("%s not equal (%.2f != %.2f)", stat, val, threshold.Val) case ">": - return fmt.Sprintf("%s %s (%.2f < %.2f)", stat, TooLow, val, threshold.Val) + return fmt.Sprintf("%s too low (%.2f < %.2f)", stat, val, threshold.Val) case "<": - return fmt.Sprintf("%s %s (%.2f > %.2f)", stat, TooHigh, val, threshold.Val) + return fmt.Sprintf("%s too high (%.2f > %.2f)", stat, val, threshold.Val) case ">=": - return fmt.Sprintf("%s %s(%.2f <= %.2f)", stat, TooLow, val, threshold.Val) + return fmt.Sprintf("%s too low (%.2f <= %.2f)", stat, val, threshold.Val) case "<=": - return fmt.Sprintf("%s %s (%.2f >= %.2f)", stat, TooHigh, val, threshold.Val) + return fmt.Sprintf("%s too high (%.2f >= %.2f)", stat, val, threshold.Val) default: return fmt.Sprintf("ERROR: Invalid Threshold: %+v", threshold) } diff --git a/traffic_monitor/manager/statecombiner.go b/traffic_monitor/manager/statecombiner.go index 55b6a5fa86..c56d86700d 100644 --- a/traffic_monitor/manager/statecombiner.go +++ b/traffic_monitor/manager/statecombiner.go @@ -128,7 +128,7 @@ func combineCacheState( IPv6Available: ipv6Available}) } - combinedStates.AddCache(cacheName, tc.IsAvailable{IsAvailable: available, Ipv4Available: ipv4Available, Ipv6Available: ipv6Available, DirectlyPolled: localCacheState.DirectlyPolled, Status: localCacheState.Status, LastPoll: localCacheState.LastPoll, LastPollV6: localCacheState.LastPollV6}) + combinedStates.AddCache(cacheName, tc.IsAvailable{IsAvailable: available, Ipv4Available: ipv4Available, Ipv6Available: ipv6Available, DirectlyPolled: localCacheState.DirectlyPolled, Status: localCacheState.Status, LastPoll: localCacheState.LastPoll}) } func combineDSState( diff --git a/traffic_monitor/todata/todata.go b/traffic_monitor/todata/todata.go index d6f8c2f903..d1b8654f34 100644 --- a/traffic_monitor/todata/todata.go +++ b/traffic_monitor/todata/todata.go @@ -78,7 +78,6 @@ type TOData struct { ServerCachegroups map[tc.CacheName]tc.CacheGroupName ServerDeliveryServices map[tc.CacheName][]tc.DeliveryServiceName ServerTypes map[tc.CacheName]tc.CacheType - SameIpServers map[tc.CacheName]map[tc.CacheName]bool } // New returns a new empty TOData object, initializing pointer members. @@ -90,7 +89,6 @@ func New() *TOData { DeliveryServiceTypes: map[tc.DeliveryServiceName]tc.DSTypeCategory{}, DeliveryServiceRegexes: NewRegexes(), ServerCachegroups: map[tc.CacheName]tc.CacheGroupName{}, - SameIpServers: map[tc.CacheName]map[tc.CacheName]bool{}, } } @@ -120,12 +118,6 @@ func (d TODataThreadsafe) set(newTOData TOData) { d.m.Unlock() } -func (d TODataThreadsafe) SetForTest(newTOData TOData) { - d.m.Lock() - *d.toData = newTOData - d.m.Unlock() -} - // CRConfig is the CrConfig data needed by TOData. Note this is not all data in the CRConfig. // TODO change strings to type? type CRConfig struct { @@ -187,12 +179,6 @@ func (d TODataThreadsafe) Update(to towrap.TrafficOpsSessionThreadsafe, cdn stri return fmt.Errorf("getting server types from monitoring config: %v", err) } - if val, ok := mc.Config["tm.sameipservers.enabled"]; ok && val.(string) == "true" { - newTOData.SameIpServers = getSameIPServers(mc) - } else { - newTOData.SameIpServers = make(map[tc.CacheName]map[tc.CacheName]bool) - } - d.set(newTOData) return nil } @@ -356,48 +342,6 @@ func getServerTypes(mc tc.TrafficMonitorConfigMap) (map[tc.CacheName]tc.CacheTyp return serverTypes, nil } -// getSameIPServers gets the caches that have the same VIP -func getSameIPServers(mc tc.TrafficMonitorConfigMap) map[tc.CacheName]map[tc.CacheName]bool { - sameIPServers := map[tc.CacheName]map[tc.CacheName]bool{} - - // get service addresses - serviceAddress := map[string][]string{} - for server, serverData := range mc.TrafficServer { - for _, intf := range serverData.Interfaces { - for _, addr := range intf.IPAddresses { - if addr.ServiceAddress { - if _, ok := serviceAddress[addr.Address]; !ok { - serviceAddress[addr.Address] = []string{} - } - serviceAddress[addr.Address] = append(serviceAddress[addr.Address], server) - } - } - } - } - - for server, serverData := range mc.TrafficServer { - for _, intf := range serverData.Interfaces { - for _, addr := range intf.IPAddresses { - if addr.ServiceAddress { - // if service addresses belongs to more than one server - if len(serviceAddress[addr.Address]) > 1 { - if _, ok := sameIPServers[tc.CacheName(server)]; !ok { - sameIPServers[tc.CacheName(server)] = map[tc.CacheName]bool{} - } - for _, partner := range serviceAddress[addr.Address] { - if partner != server { - sameIPServers[tc.CacheName(server)][tc.CacheName(partner)] = true - } - } - } - } - } - } - } - - return sameIPServers -} - // canUseMonitorConfig returns true if we can prefer monitor config data to crconfig data. func canUseMonitorConfig(mc tc.TrafficMonitorConfigMap) bool { for _, dsData := range mc.DeliveryService { diff --git a/traffic_monitor/todata/todata_test.go b/traffic_monitor/todata/todata_test.go index 14c56ee128..e28c38019e 100644 --- a/traffic_monitor/todata/todata_test.go +++ b/traffic_monitor/todata/todata_test.go @@ -142,71 +142,3 @@ func TestGetDeliveryServiceServersWithNonTopologyBasedDeliveryService(t *testing t.Fatalf("getDeliveryServiceServers with non-topology-based delivery service expected: %+v actual: %+v", expectedNonTopologiesTOData, nonTopologiesTOData) } } - -func TestGetSameIPServers(t *testing.T) { - mc := tc.TrafficMonitorConfigMap{TrafficServer: make(map[string]tc.TrafficServer)} - mc.TrafficServer["server1_ip1"] = tc.TrafficServer{ - Interfaces: []tc.ServerInterfaceInfo{ - { - IPAddresses: []tc.ServerIPAddress{ - {Address: "10.0.0.1", ServiceAddress: true}, - }, - }, - }, - } - mc.TrafficServer["server2_ip1"] = tc.TrafficServer{ - Interfaces: []tc.ServerInterfaceInfo{ - { - IPAddresses: []tc.ServerIPAddress{ - {Address: "10.0.0.1", ServiceAddress: true}, - }, - }, - }, - } - mc.TrafficServer["server4_ip1_no_service"] = tc.TrafficServer{ - Interfaces: []tc.ServerInterfaceInfo{ - { - IPAddresses: []tc.ServerIPAddress{ - {Address: "10.0.0.1"}, - }, - }, - }, - } - mc.TrafficServer["server3_ip3"] = tc.TrafficServer{ - Interfaces: []tc.ServerInterfaceInfo{ - { - IPAddresses: []tc.ServerIPAddress{ - {Address: "10.0.0.3", ServiceAddress: true}, - }, - }, - }, - } - sameIpServers := getSameIPServers(mc) - if _, ok := sameIpServers[("server1_ip1")]; !ok { - t.Fatal("getSameIPServers expected to find server1_ip1") - } - if _, ok := sameIpServers["server1_ip1"]["server2_ip1"]; !ok { - t.Fatal("getSameIPServers expected to find server1_ip1 to have same ip as server2_ip1") - } - if _, ok := sameIpServers["server2_ip1"]; !ok { - t.Fatal("getSameIPServers expected to find server2_ip1") - } - if _, ok := sameIpServers["server1_ip1"]["server2_ip1"]; !ok { - t.Fatal("getSameIPServers expected to find server2_ip1 to have same ip as server1_ip1") - } - if _, ok := sameIpServers["server1_ip1"]["server4_ip1_no_service"]; ok { - t.Fatal("getSameIPServers expected to find server1_ip1 not to have same ip as server4_ip1_no_service") - } - if _, ok := sameIpServers["server3_ip3"]; ok { - t.Fatal("getSameIPServers expected to not find server3_ip3") - } - - expectedSameIpServers := map[string]map[string]bool{} - expectedSameIpServers["server1_ip1"] = map[string]bool{"server2_ip1": true} - expectedSameIpServers["server2_ip1"] = map[string]bool{"server1_ip1": true} - - if !reflect.DeepEqual(expectedSameIpServers, sameIpServers) { - - } - -}
