ocket8888 opened a new issue #4946:
URL: https://github.com/apache/trafficcontrol/issues/4946
## I'm submitting a ...
- bug report
## Traffic Control components affected ...
- Traffic Monitor
## Current behavior:
When you try to get Traffic Monitor to monitor caches using backup, local
file copies of the CDN snapshot and `cdns/{{name}}/configs/monitoring` payload,
it will never actually monitor anything and just report that its API is
perpetually unavailable because the "service is still starting".
## Expected / new behavior:
backup files for the snapshot and/or monitoring payload should enable the
monitor to work in the absence of Traffic Ops.
## Minimal reproduction of the problem with instructions:
Run Traffic Monitor with no Traffic Ops, but with back-up files.
## Anything else:
<details><summary>Log files</summary>
```
INFO: traffic_monitor.go:69: 2020-08-12T10:54:36.326919133-06:00: Starting
with config {CacheHealthPollingInterval:6s CacheStatPollingInterval:6s
MonitorConfigPollingInterval:5s HTTPTimeout:2s PeerPollingInterval:5s
PeerOptimistic:true PeerOptimisticQuorumMin:0 MaxEvents:200 MaxStatHistory:5
MaxHealthHistory:5 HealthFlushInterval:20ms StatFlushInterval:20ms
StatBufferInterval:0s LogLocationError:stderr LogLocationWarning:stderr
LogLocationInfo:stderr LogLocationDebug:stderr LogLocationEvent:stderr
ServeReadTimeout:10s ServeWriteTimeout:10s HealthToStatRatio:4
StaticFileDir:/home/ocket8888/src/github.com/apache/trafficcontrol/traffic_monitor/static/
CRConfigHistoryCount:20000 TrafficOpsMinRetryInterval:100ms
TrafficOpsMaxRetryInterval:1m0s
CRConfigBackupFile:/home/ocket8888/src/github.com/apache/trafficcontrol/traffic_monitor/snapshot.json
TMConfigBackupFile:/home/ocket8888/src/github.com/apache/trafficcontrol/traffic_monitor/monitoring.json
TrafficOpsDiskRetryMax:2 CachePollingPro
tocol:both PeerPollingProtocol:both}
INFO: srvhttp.go:95: 2020-08-12T10:54:36.335072547-06:00: Starting Web Server
INFO: srvhttp.go:142: 2020-08-12T10:54:36.336100084-06:00: Web server
listening on :8080
ERROR: opsconfig.go:81: 2020-08-12T10:54:36.378264236-06:00:
OpsConfigManager: MonitorConfigPoller: error instantiating Session with
traffic_ops (<nil>): logging in: requesting: Post
"https://localhos:6443/api/2.0/user/login": dial tcp: lookup localhos:
Temporary failure in name resolution
ERROR: opsconfig.go:176: 2020-08-12T10:54:36.37828741-06:00: retrying in
100ms
ERROR: opsconfig.go:81: 2020-08-12T10:54:36.515605707-06:00:
OpsConfigManager: MonitorConfigPoller: error instantiating Session with
traffic_ops (<nil>): logging in: requesting: Post
"https://localhos:6443/api/2.0/user/login": dial tcp: lookup localhos:
Temporary failure in name resolution
ERROR: opsconfig.go:176: 2020-08-12T10:54:36.515629332-06:00: retrying in
256.709987ms
ERROR: opsconfig.go:81: 2020-08-12T10:54:36.808954271-06:00:
OpsConfigManager: MonitorConfigPoller: error instantiating Session with
traffic_ops (<nil>): logging in: requesting: Post
"https://localhos:6443/api/2.0/user/login": dial tcp: lookup localhos:
Temporary failure in name resolution
ERROR: opsconfig.go:176: 2020-08-12T10:54:36.808977503-06:00: retrying in
418.581246ms
ERROR: opsconfig.go:195: 2020-08-12T10:54:37.227765571-06:00: error
instantiating Session with traffic_ops, backup disk files exist, creating empty
traffic_ops session to read
ERROR: opsconfig.go:81: 2020-08-12T10:54:37.267767065-06:00:
OpsConfigManager: getting CDN name from Traffic Ops, using config CDN
'CDN-in-a-Box': getting monitor pop-os CDN: Get
"https://localhos:6443/api/2.0/servers": dial tcp: lookup localhos: Temporary
failure in name resolution
ERROR: towrap.go:284: 2020-08-12T10:54:37.306591952-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
INFO: monitorconfig.go:92: 2020-08-12T10:54:37.307895342-06:00:
MonitorConfigPoller: received new session: {0xc00038e250 0xc0003200b8
{0xc00038e258 0xc00038ce00} {0xc0003890e0 0xc0003220a0 0xc00038aa18
0xc000320048 0xc0003200b0}
/home/ocket8888/src/github.com/apache/trafficcontrol/traffic_monitor/snapshot.json
/home/ocket8888/src/github.com/apache/trafficcontrol/traffic_monitor/monitoring.json}
INFO: monitorconfig.go:89: 2020-08-12T10:54:37.308208414-06:00:
MonitorConfigPoller: received new opsConfig: {admin twelve12
https://localhos:6443 true CDN-in-a-Box :8080 }
ERROR: towrap.go:351: 2020-08-12T10:54:41.367882457-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:41.412689169-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
ERROR: monitorconfig.go:256: 2020-08-12T10:54:41.412863011-06:00: monitor
config server edge profile ATS_EDGE_TIER_CACHE has no polling URL; can't poll
ERROR: monitorconfig.go:256: 2020-08-12T10:54:41.412872111-06:00: monitor
config server mid profile ATS_MID_TIER_CACHE has no polling URL; can't poll
ERROR: monitorconfig.go:317: 2020-08-12T10:54:41.412883222-06:00: No
REPORTED caches exist in Traffic Ops, nothing to poll.
INFO: monitorconfig.go:98: 2020-08-12T10:54:41.412889529-06:00:
MonitorConfigPoller: received new interval: 1.94s
ERROR: towrap.go:351: 2020-08-12T10:54:43.394142444-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:43.430899146-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:351: 2020-08-12T10:54:45.325607767-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:45.354990392-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:54:46.082734416-06:00: poll 1 2020-08-12
10:54:46.082733489 -0600 MDT m=+9.759539641 start
DEBUG: cache.go:195: 2020-08-12T10:54:46.090571056-06:00: poll 1 2020-08-12
10:54:46.090570638 -0600 MDT m=+9.767376792 poller end
1597251286.091 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:54:47.275805677-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:47.311117606-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:54:48.992830949-06:00: poll 2 2020-08-12
10:54:48.992829863 -0600 MDT m=+12.669636014 start
DEBUG: cache.go:195: 2020-08-12T10:54:48.993747282-06:00: poll 2 2020-08-12
10:54:48.993746983 -0600 MDT m=+12.670553134 poller end
1597251288.994 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:54:49.210830419-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:49.248258005-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:351: 2020-08-12T10:54:51.152823319-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:51.189231418-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:54:51.902793887-06:00: poll 3 2020-08-12
10:54:51.902792678 -0600 MDT m=+15.579598830 start
DEBUG: cache.go:195: 2020-08-12T10:54:51.903425225-06:00: poll 3 2020-08-12
10:54:51.903424684 -0600 MDT m=+15.580230835 poller end
1597251291.903 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:54:53.09172033-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:53.127016769-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:54:54.812863263-06:00: poll 4 2020-08-12
10:54:54.812862327 -0600 MDT m=+18.489668483 start
DEBUG: cache.go:195: 2020-08-12T10:54:54.813805086-06:00: poll 4 2020-08-12
10:54:54.813804224 -0600 MDT m=+18.490610378 poller end
1597251294.814 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:54:55.03419012-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:55.070193445-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:351: 2020-08-12T10:54:56.97234731-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:57.010424642-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:54:57.722868313-06:00: poll 5 2020-08-12
10:54:57.722867191 -0600 MDT m=+21.399673342 start
DEBUG: cache.go:195: 2020-08-12T10:54:57.723315703-06:00: poll 5 2020-08-12
10:54:57.723315389 -0600 MDT m=+21.400121539 poller end
1597251297.723 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:54:58.914038703-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:54:58.952297567-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:55:00.632803602-06:00: poll 6 2020-08-12
10:55:00.632801621 -0600 MDT m=+24.309607965 start
DEBUG: cache.go:195: 2020-08-12T10:55:00.633457964-06:00: poll 6 2020-08-12
10:55:00.633457633 -0600 MDT m=+24.310263785 poller end
1597251300.633 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:55:00.854219004-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:55:00.90142851-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:351: 2020-08-12T10:55:02.78482135-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:55:02.815255072-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:55:03.542874474-06:00: poll 7 2020-08-12
10:55:03.542872959 -0600 MDT m=+27.219679120 start
DEBUG: cache.go:195: 2020-08-12T10:55:03.543484308-06:00: poll 7 2020-08-12
10:55:03.543455767 -0600 MDT m=+27.220261918 poller end
1597251303.543 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:55:04.736390267-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:55:04.774975785-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:55:06.452881216-06:00: poll 8 2020-08-12
10:55:06.452879981 -0600 MDT m=+30.129686143 start
DEBUG: cache.go:195: 2020-08-12T10:55:06.453547047-06:00: poll 8 2020-08-12
10:55:06.453546552 -0600 MDT m=+30.130352709 poller end
1597251306.454 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:55:06.66971386-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:55:06.707254065-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:351: 2020-08-12T10:55:08.609784165-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:55:08.639473066-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:55:09.362821492-06:00: poll 9 2020-08-12
10:55:09.362820293 -0600 MDT m=+33.039626444 start
DEBUG: cache.go:195: 2020-08-12T10:55:09.363342284-06:00: poll 9 2020-08-12
10:55:09.363342025 -0600 MDT m=+33.040148176 poller end
1597251309.363 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:55:10.55142877-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:55:10.589349233-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
DEBUG: cache.go:183: 2020-08-12T10:55:12.272885175-06:00: poll 10 2020-08-12
10:55:12.272883975 -0600 MDT m=+35.949690125 start
DEBUG: cache.go:195: 2020-08-12T10:55:12.280506939-06:00: poll 10 2020-08-12
10:55:12.280506316 -0600 MDT m=+35.957312464 poller end
1597251312.281 host="trafficmonitor", type=PEER, available=true, msg="Peer
is reachable"
ERROR: towrap.go:351: 2020-08-12T10:55:12.48937712-06:00: Error getting
configMap from traffic_ops, backup file exists, reading from file
ERROR: towrap.go:284: 2020-08-12T10:55:12.530578105-06:00: Error getting
CRConfig from traffic_ops, backup file exists, reading from file
```
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
