This is an automated email from the ASF dual-hosted git repository.
dgelinas pushed a commit to branch 3.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
The following commit(s) were added to refs/heads/3.0.x by this push:
new 21a5491 Fix TM polling urls with ports (#3605) (#3642)
21a5491 is described below
commit 21a549189543a458a0ec8e5d24c0d22389dc6255
Author: Robert Butts <[email protected]>
AuthorDate: Wed May 29 12:17:37 2019 -0600
Fix TM polling urls with ports (#3605) (#3642)
* Fix TM polling urls with ports
* Add Changelog issue number '#'
* Add Docs terms for monitor poll url info
(cherry picked from commit 3871f80bba888bcca535d84d8638cd20e8633fe7)
---
CHANGELOG.md | 2 +-
docs/source/admin/traffic_monitor.rst | 27 +++++++
lib/go-tc/traffic_router.go | 1 +
traffic_monitor/manager/monitorconfig.go | 64 +++++++++++-----
traffic_monitor/manager/monitorconfig_test.go | 102 ++++++++++++++++++++++++++
traffic_monitor/towrap/towrap.go | 5 ++
6 files changed, 183 insertions(+), 18 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d0526fa..956eefa 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -24,7 +24,7 @@ The format is based on [Keep a
Changelog](http://keepachangelog.com/en/1.0.0/).
- Correct FQDN case mismatch when generating DNSSEC.
- Issue 3223 - cannot add steering targets to steering delivery service.
- Issue 3466 - docs fail to build in python 3.6.
-
+- Issue #3605: Fixed Traffic Monitor custom ports in health polling URL.
## [3.0.0] - 2019-02-13
### Added
diff --git a/docs/source/admin/traffic_monitor.rst
b/docs/source/admin/traffic_monitor.rst
index 98b0080..fd4fc88 100755
--- a/docs/source/admin/traffic_monitor.rst
+++ b/docs/source/admin/traffic_monitor.rst
@@ -55,6 +55,33 @@ Configuration Overview
Traffic Monitor is configured via two JSON configuration files,
``traffic_ops.cfg`` and ``traffic_monitor.cfg``, by default located in the
``conf`` directory in the install location. ``traffic_ops.cfg`` contains
Traffic Ops connection information. Specify the URL, username, and password for
the instance of Traffic Ops of which this Traffic Monitor is a member.
``traffic_monitor.cfg`` contains log file locations, as well as detailed
application configuration variables such as processing f [...]
+Cache Polling URL
+-----------------------------------
+
+The :term:`cache servers` are polled at the URL specified in the
``health.polling.url`` :term:`parameter`, on the :term:`cache server`'s
:term:`profile`.
+
+This :term:`parameter` must have the config file ``rascal.properties``.
+
+The value is a template with the text ``${hostname}`` being replaced with the
:term:`cache server`'s Network IP (IPv4), and ``${interface_name}`` being
replaced with the :term:`cache server`'s network Interface Name. For example,
``http://${hostname}/_astats?application=&inf.name=${interface_name}``.
+
+If the template contains a port, that port will be used, and the :term:`cache
server`'s HTTPS and TCP Ports will not be added.
+
+If the template does not contain a port, then if the template starts with
``https`` the :term:`cache server`'s HTTPS Port will be added, and if the
template doesn't start with ``https`` the :term:`cache server`'s TCP Port will
be added.
+
+Examples:
+
+Template
``http://${hostname}/_astats?application=&inf.name=${interface_name}`` Server
IP ``192.0.2.42`` Server TCP Port ``8080`` HTTPS Port ``8443`` becomes
``http://192.0.2.42:8080/_astats?application=&inf.name=${interface_name}``.
+Template
``https://${hostname}/_astats?application=&inf.name=${interface_name}`` Server
IP ``192.0.2.42`` Server TCP Port ``8080`` HTTPS Port ``8443`` becomes
``https://192.0.2.42:8443/_astats?application=&inf.name=${interface_name}``.
+Template
``http://${hostname}:1234/_astats?application=&inf.name=${interface_name}``
Server IP ``192.0.2.42`` Server TCP Port ``8080`` HTTPS Port ``8443`` becomes
``http://192.0.2.42:1234/_astats?application=&inf.name=${interface_name}``.
+Template
``https://${hostname}:1234/_astats?application=&inf.name=${interface_name}``
Server IP ``192.0.2.42`` Server TCP Port ``8080`` HTTPS Port ``8443`` becomes
``https://192.0.2.42:1234/_astats?application=&inf.name=${interface_name}``.
+
+Stat and Health Flush Configuration
+-----------------------------------
+The Monitor has a health flush interval, a stat flush interval, and a stat
buffer interval. Recall that the monitor polls both stats and health. The
health poll is so small and fast, a buffer is largely unnecessary. However, in
a large CDN, the stat poll may involve thousands of :term:`cache server` s with
thousands of stats each, or more, and CPU may be a bottleneck.
+
+The flush intervals, ``health_flush_interval_ms`` and
``stat_flush_interval_ms``, indicate how often to flush stats or health, if
results are continuously coming in with no break. This prevents starvation.
Ideally, if there is enough CPU, the flushes should never occur. The default
flush times are 200 milliseconds, which is suggested as a reasonable starting
point; operators may adjust them higher or lower depending on the need to get
health data and stop directing client traffic to unhe [...]
+
+The stat buffer interval, ``stat_buffer_interval_ms``, also provides a
temporal buffer for stat processing. Stats will not be processed except after
this interval, whereupon all pending stats will be processed, unless the flush
interval occurs as a starvation safety. The stat buffer and flush intervals may
be thought of as a state machine with two states: the "buffer state" accepts
results until the buffer interval has elapsed, whereupon the "flush state" is
entered, and results are acce [...]
Troubleshooting and Log Files
=============================
diff --git a/lib/go-tc/traffic_router.go b/lib/go-tc/traffic_router.go
index f636f7a..554f64b 100644
--- a/lib/go-tc/traffic_router.go
+++ b/lib/go-tc/traffic_router.go
@@ -136,6 +136,7 @@ type TrafficServer struct {
CacheGroup string `json:"cacheGroup"`
IP6 string `json:"ip6"`
Port int `json:"port"`
+ HTTPSPort int `json:"httpsPort,omitempty"`
HostName string `json:"hostName"`
FQDN string `json:"fqdn"`
InterfaceName string `json:"interfaceName"`
diff --git a/traffic_monitor/manager/monitorconfig.go
b/traffic_monitor/manager/monitorconfig.go
index ab0bfbc..341c743 100644
--- a/traffic_monitor/manager/monitorconfig.go
+++ b/traffic_monitor/manager/monitorconfig.go
@@ -21,6 +21,7 @@ package manager
import (
"fmt"
+ "net/url"
"os"
"strconv"
"strings"
@@ -250,8 +251,8 @@ func monitorConfigListen(
localStates.AddCache(cacheName,
tc.IsAvailable{IsAvailable: false})
}
- url :=
monitorConfig.Profile[srv.Profile].Parameters.HealthPollingURL
- if url == "" {
+ pollURLStr :=
monitorConfig.Profile[srv.Profile].Parameters.HealthPollingURL
+ if pollURLStr == "" {
log.Errorf("monitor config server %v profile %v
has no polling URL; can't poll", srv.HostName, srv.Profile)
continue
}
@@ -268,18 +269,7 @@ func monitorConfigListen(
log.Infof("health.polling.type for '%v' is
empty, using default '%v'", srv.HostName, pollType)
}
- port := ""
- if srv.Port != 0 {
- port = ":" + strconv.Itoa(srv.Port)
- }
-
- r := strings.NewReplacer(
- "${hostname}", srv.IP+port,
- "${interface_name}", srv.InterfaceName,
- "application=plugin.remap",
"application=system",
- "application=", "application=system",
- )
- url = r.Replace(url)
+ pollURLStr = createServerHealthPollURL(pollURLStr, srv)
connTimeout :=
trafficOpsHealthConnectionTimeoutToDuration(monitorConfig.Profile[srv.Profile].Parameters.HealthConnectionTimeout)
if connTimeout == 0 {
@@ -287,9 +277,9 @@ func monitorConfigListen(
log.Warnln("profile " + srv.Profile + "
health.connection.timeout Parameter is missing or zero, using default " +
DefaultHealthConnectionTimeout.String())
}
- healthURLs[srv.HostName] = poller.PollConfig{URL: url,
Host: srv.FQDN, Timeout: connTimeout, Format: format, PollType: pollType}
- r = strings.NewReplacer("application=system",
"application=")
- statURL := r.Replace(url)
+ healthURLs[srv.HostName] = poller.PollConfig{URL:
pollURLStr, Host: srv.FQDN, Timeout: connTimeout, Format: format, PollType:
pollType}
+
+ statURL := createServerStatPollURL(pollURLStr)
statURLs[srv.HostName] = poller.PollConfig{URL:
statURL, Host: srv.FQDN, Timeout: connTimeout, Format: format, PollType:
pollType}
}
@@ -341,3 +331,43 @@ func monitorConfigListen(
}
}
}
+
+// createServerHealthPollURL takes the template pollingURLStr, and replaces
variables with data from srv, and returns the polling URL for srv.
+func createServerHealthPollURL(pollingURLStr string, srv tc.TrafficServer)
string {
+ pollingURLStr = strings.NewReplacer(
+ "${hostname}", srv.IP,
+ "${interface_name}", srv.InterfaceName,
+ "application=plugin.remap", "application=system",
+ "application=", "application=system",
+ ).Replace(pollingURLStr)
+
+ if strings.HasPrefix(strings.ToLower(pollingURLStr), "https") {
+ if srv.HTTPSPort != 0 {
+ pollURL, err := url.Parse(pollingURLStr)
+ if err != nil {
+ log.Warnln("profile " + srv.Profile + " cache
'" + srv.FQDN + "' polling URL '" + pollingURLStr + "' failed to parse, may not
be a valid URL! Using anyway, not using custom HTTPS Port " +
strconv.Itoa(srv.HTTPSPort) + "!")
+ } else if pollURL.Port() == "" { // if there's both an
HTTPS Port and a port in the polling URL, the polling URL takes precedence
+ pollURL.Host += ":" +
strconv.Itoa(srv.HTTPSPort)
+ pollingURLStr = pollURL.String()
+ }
+ }
+ } else {
+ if srv.Port != 0 {
+ pollURL, err := url.Parse(pollingURLStr)
+ if err != nil {
+ log.Warnln("profile " + srv.Profile + " cache
'" + srv.FQDN + "' polling URL '" + pollingURLStr + "' failed to parse, may not
be a valid URL! Using anyway, not using custom TCP Port " +
strconv.Itoa(srv.Port) + "!")
+ } else if pollURL.Port() == "" { // if there's both a
TCP Port and a port in the polling URL, the polling URL takes precedence
+ pollURL.Host += ":" + strconv.Itoa(srv.Port)
+ pollingURLStr = pollURL.String()
+ }
+ }
+ }
+
+ return pollingURLStr
+}
+
+// createServerStatPollURL takes the health polling URL string, and modifies
it to be the stat poll URL.
+// Note this does not replace template variables with server values,
healthPollURLStr must be the health URL for a given server, not a template.
+func createServerStatPollURL(healthPollURLStr string) string {
+ return strings.NewReplacer("application=system",
"application=").Replace(healthPollURLStr)
+}
diff --git a/traffic_monitor/manager/monitorconfig_test.go
b/traffic_monitor/manager/monitorconfig_test.go
new file mode 100644
index 0000000..5d5b145
--- /dev/null
+++ b/traffic_monitor/manager/monitorconfig_test.go
@@ -0,0 +1,102 @@
+package manager
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "strconv"
+ "testing"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+func TestCreateServerHealthPollURL(t *testing.T) {
+ tmpl :=
`http://${hostname}/_astats?application=&inf.name=${interface_name}`
+ srv := tc.TrafficServer{IP: "192.0.2.42", InterfaceName: "george"}
+
+ expected := `http://` + srv.IP +
`/_astats?application=system&inf.name=` + srv.InterfaceName
+ actual := createServerHealthPollURL(tmpl, srv)
+
+ if expected != actual {
+ t.Errorf("expected createServerHealthPollURL '" + expected + "'
actual: '" + actual + "'")
+ }
+}
+
+func TestCreateServerHealthPollURLTemplatePort(t *testing.T) {
+ tmpl :=
`http://${hostname}:1234/_astats?application=&inf.name=${interface_name}`
+ srv := tc.TrafficServer{IP: "192.0.2.42", InterfaceName: "george"}
+
+ expected := `http://` + srv.IP +
`:1234/_astats?application=system&inf.name=` + srv.InterfaceName
+ actual := createServerHealthPollURL(tmpl, srv)
+
+ if expected != actual {
+ t.Errorf("expected createServerHealthPollURL '" + expected + "'
actual: '" + actual + "'")
+ }
+}
+
+func TestCreateServerHealthPollURLServerPort(t *testing.T) {
+ tmpl :=
`http://${hostname}/_astats?application=&inf.name=${interface_name}`
+ srv := tc.TrafficServer{IP: "192.0.2.42", Port: 5678, HTTPSPort: 910,
InterfaceName: "george"}
+
+ expected := `http://` + srv.IP + ":" + strconv.Itoa(srv.Port) +
`/_astats?application=system&inf.name=` + srv.InterfaceName
+ actual := createServerHealthPollURL(tmpl, srv)
+
+ if expected != actual {
+ t.Errorf("expected createServerHealthPollURL '" + expected + "'
actual: '" + actual + "'")
+ }
+}
+
+func TestCreateServerHealthPollURLServerPortHTTPS(t *testing.T) {
+ tmpl :=
`hTTps://${hostname}/_astats?application=&inf.name=${interface_name}`
+ srv := tc.TrafficServer{IP: "192.0.2.42", Port: 5678, HTTPSPort: 910,
InterfaceName: "george"}
+
+ expected := `https://` + srv.IP + ":" + strconv.Itoa(srv.HTTPSPort) +
`/_astats?application=system&inf.name=` + srv.InterfaceName
+ actual := createServerHealthPollURL(tmpl, srv)
+
+ if expected != actual {
+ t.Errorf("expected createServerHealthPollURL '" + expected + "'
actual: '" + actual + "'")
+ }
+}
+
+func TestCreateServerHealthPollURLTemplateAndServerPort(t *testing.T) {
+
+ // if both template and server ports exist, template takes precedence
+
+ tmpl :=
`http://${hostname}:1234/_astats?application=&inf.name=${interface_name}`
+ srv := tc.TrafficServer{IP: "192.0.2.42", Port: 5678, HTTPSPort: 910,
InterfaceName: "george"}
+
+ expected := `http://` + srv.IP +
`:1234/_astats?application=system&inf.name=` + srv.InterfaceName
+ actual := createServerHealthPollURL(tmpl, srv)
+
+ if expected != actual {
+ t.Errorf("expected createServerHealthPollURL '" + expected + "'
actual: '" + actual + "'")
+ }
+}
+
+func TestCreateServerStatPollURL(t *testing.T) {
+ tmpl :=
`http://${hostname}/_astats?application=&inf.name=${interface_name}`
+ srv := tc.TrafficServer{IP: "192.0.2.42", InterfaceName: "george"}
+
+ expected := `http://` + srv.IP + `/_astats?application=&inf.name=` +
srv.InterfaceName
+ actual := createServerStatPollURL(createServerHealthPollURL(tmpl, srv))
+
+ if expected != actual {
+ t.Errorf("expected createServerStatPollURL '" + expected + "'
actual: '" + actual + "'")
+ }
+}
diff --git a/traffic_monitor/towrap/towrap.go b/traffic_monitor/towrap/towrap.go
index 55e80e3..76dbedf 100644
--- a/traffic_monitor/towrap/towrap.go
+++ b/traffic_monitor/towrap/towrap.go
@@ -351,6 +351,11 @@ func CreateMonitorConfig(crConfig tc.CRConfig, mc
*tc.TrafficMonitorConfigMap) (
} else {
log.Warnf("Creating monitor config: CRConfig server %s
missing HashId field\n", name)
}
+ if srv.HttpsPort != nil {
+ s.HTTPSPort = *srv.HttpsPort
+ } else {
+ log.Warnf("Creating monitor config: CRConfig server %s
missing HttpsPort field\n", name)
+ }
mc.TrafficServer[name] = s
}
