[GitHub] incubator-trafficcontrol pull request #852: Add TO Go wrapper to return auth...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/852 Add TO Go wrapper to return auth data You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-go-wrapauthdata Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/852.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #852 commit f73847108befb105c4305631772618551f5d005c Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-22T19:50:45Z Add TO Go wrapper to return auth data --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #849: Fix Traffic Ops service status t...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/849 Fix Traffic Ops service status to include Go process You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-go-servicestatus Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/849.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #849 commit a299744271cfc547badffde3c3d371990bd9aacd Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-22T04:18:35Z Fix TO Go service status to include Go process --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #843: Add TM log EventRaw, for perform...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/843 Add TM log EventRaw, for performance. Not using Printf is drastically faster for frequent logs, like Events. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol tm-logeventraw Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/843.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #843 commit 88ab4dd2e179d54f281cc0479414555c36cf9392 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-21T20:38:42Z Add TM log EventRaw, for performance. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #829: Fix TO Go monitoring numeric val...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/829 Fix TO Go monitoring numeric values Perl TO monitoring.json returns numeric types for parameters whose strings successfully convert to integers. This replicates that behavior. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-go-monitoringtypes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/829.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #829 commit 0551be2cbe732de00be5cbb2da350b752b5ffdde Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-16T23:56:53Z Fix TO Go monitoring numeric values --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #827: Fix Traffic Ops Go cookie to hav...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/827 Fix Traffic Ops Go cookie to have path, httponly You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-go-fixcookiepath Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/827.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #827 commit c022f7600ebc045aad372d536074c0a3e017a128 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-16T22:01:03Z Fix TO Go cookie to have path, httponly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #826: Fix Traffic Monitor log EventfRa...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/826 Fix Traffic Monitor log EventfRaw for nil loggers You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol tm-log-fixeventfraw Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/826.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #826 commit fab62e38ac2bcdf1babe3b3194f4673a00e5d4a4 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-16T21:59:44Z Fix TM log EventfRaw for nil loggers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #801: Add Traffic Ops Golang Endpoint Convers...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/801 I will, just haven't had time. Hopefully this weekend. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #810: Change ORT Integrity Check to fa...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/810 Change ORT Integrity Check to fall back to Content-Length Changes ORT to do a Message Integrity Check with Content-Length if it exists and Whole-Content-SHA512 doesn't. This specifically allows newer versions of ORT to work with older versions of Traffic Ops. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-ort-miccontentlengthfallback Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/810.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #810 commit ca8e26672e0c96615748e172097927afbd8aa271 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-15T00:17:04Z Change TO ORT MIC to fall back to Content-Length Changes ORT to do a Message Integry Check with Content-Length if it exists and Whole-Content-SHA512 doesn't. This specifically allows newer versions of ORT to work with older versions of Traffic Ops. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #786: Fix TO ORT for missing Content-Length, ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/786 Has been merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #786: Fix TO ORT for missing Content-L...
Github user rob05c closed the pull request at: https://github.com/apache/incubator-trafficcontrol/pull/786 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #800: Add Traffic Ops Golang priv leve...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/800 Add Traffic Ops Golang priv level constants You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-go-privlevelconsts Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/800.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #800 commit f603df2ed5b46d748f959b93d8cb695f4aee8617 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-13T18:49:55Z Add TO Go priv level constants --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #786: Fix TO ORT for missing Content-L...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/786#discussion_r132784235
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -1623,9 +1623,7 @@ sub check_lwp_response_content_length {
my $url = $lwp_response->request->uri;
if ( !defined($lwp_response->header('Content-Length')) ) {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url
did not return a Content-Length header!\n";
- exit;
- return 1;
+ return 0; # Content-Length MAY be omitted per HTTP/1.1 RFC
7230, and in fact MUST NOT be included with a 'Transfer-Encoding: Chunked'
header, which MUST be accepted by clients.
--- End diff --
Agreed, something like CityHash would be better. But I don't want to deal
with finding a Perl CityHash library. I'd vote Sha512 for now, and maybe switch
to CityHash/Murmur/etc when TO is completely in Go.
On the other hand, for clients, it _is_ more convenient to have a more
common hash like Sha or MD5, easier to find libraries.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #786: Fix TO ORT for missing Content-L...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/786#discussion_r132727173
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -1623,9 +1623,7 @@ sub check_lwp_response_content_length {
my $url = $lwp_response->request->uri;
if ( !defined($lwp_response->header('Content-Length')) ) {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url
did not return a Content-Length header!\n";
- exit;
- return 1;
+ return 0; # Content-Length MAY be omitted per HTTP/1.1 RFC
7230, and in fact MUST NOT be included with a 'Transfer-Encoding: Chunked'
header, which MUST be accepted by clients.
--- End diff --
X prefixes are deprecated https://tools.ietf.org/html/rfc6648
Further, X-prefixed or not, it doesn't solve the problem the deprecation
addressed, of allowing inconsistent adoption.
https://trac.ietf.org/trac/httpbis/ticket/178
How about `Whole-Content-SHA512`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #786: Fix TO ORT for missing Content-L...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/786#discussion_r131973278
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -1623,9 +1623,7 @@ sub check_lwp_response_content_length {
my $url = $lwp_response->request->uri;
if ( !defined($lwp_response->header('Content-Length')) ) {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url
did not return a Content-Length header!\n";
- exit;
- return 1;
+ return 0; # Content-Length MAY be omitted per HTTP/1.1 RFC
7230, and in fact MUST NOT be included with a 'Transfer-Encoding: Chunked'
header, which MUST be accepted by clients.
--- End diff --
Ah, `Content-MD5` was removed from HTTP/1.1 in
https://tools.ietf.org/html/rfc7231#appendix-B . It's still legal to use,
though, or we could use a custom `X-Content-MD5`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #786: Fix TO ORT for missing Content-L...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/786#discussion_r131970799
--- Diff: traffic_ops/bin/traffic_ops_ort.pl ---
@@ -1623,9 +1623,7 @@ sub check_lwp_response_content_length {
my $url = $lwp_response->request->uri;
if ( !defined($lwp_response->header('Content-Length')) ) {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url
did not return a Content-Length header!\n";
- exit;
- return 1;
+ return 0; # Content-Length MAY be omitted per HTTP/1.1 RFC
7230, and in fact MUST NOT be included with a 'Transfer-Encoding: Chunked'
header, which MUST be accepted by clients.
--- End diff --
If we still require that check. I don't think there's a way to be
compliant, while still requiring an integrity check via `Content-Length`.
It looks like the proper way to do that is via a `Content-MD5` header:
https://tools.ietf.org/html/rfc2616#section-14.15
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 Must be merged after #786 which fixes an ORT bug this exposes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #786: Fix TO ORT for missing Content-L...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/786 Fix TO ORT for missing Content-Length, per RFC Content-Length MAY be omitted per HTTP/1.1 RFC 7230, and in fact MUST NOT be included with a 'Transfer-Encoding: Chunked' header, which MUST be accepted by clients. Fixes TC-503 You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-ortcontentlengthfix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/786.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #786 commit a4815d639a46e72c8236077a28a29677c9f14c8a Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-08T16:21:00Z Fix TO ORT for missing Content-Length, per RFC Content-Length MAY be omitted per HTTP/1.1 RFC 7230, and in fact MUST NOT be included with a 'Transfer-Encoding: Chunked' header, which MUST be accepted by clients. Fixes TC-503 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #780: Traffic Ops Golang cacheconfig
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/780 Traffic Ops Golang cacheconfig Adds Traffic Ops next-generation cache/config endpoint. Dependent on #729 - must be merged after it. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-gomonitoring-cacheconfig Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/780.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #780 commit 4b20322b3c81a42e992d67793bfcec949ad33a2f Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-09T03:02:56Z Add experimental Go TO proxying old Perl app commit 40fae0728014a5b83a948018e20b3e3c7afcf486 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-09T15:22:28Z Move TO Golang microservice out of experimental commit 01d358dff77818eaf23a33b6695661f9c65e5751 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-09T15:23:49Z Add traffic_ops_golang to RPM, service commit 1cfcae457d7c6e4bc9b27a9d03a6e34b9713d00b Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-18T03:53:50Z Vendor TO GOlang go-sqlmock commit 993c536be488a28f2451a8facc351067c4b98e73 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-18T03:55:35Z Add TO Golang monitoring test commit b71c3ecb612ae56a0806f5ae2b7304bf9eae9ecc Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-18T04:17:41Z Add TO Golang Apache license headers commit ff86a195f613a7f8fa54f5a772ea53a010b9ff79 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-22T00:14:38Z Add TO Golang configurable logging commit 580894290c5911bd53c54cdb2d8aa01284d2517c Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-22T00:16:56Z Move TO tocookie out of experimental commit 05186857b7faebdcb785447a2f621f85e42fbf40 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-22T00:59:16Z Add TO Golang server name header commit 09f8652a6002dec3c0ed4443525d405afcdafce5 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-22T01:03:14Z Add TO tocookie generated-by data commit 5ed6353119eee0056bd3687af47d0776014045c9 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-22T01:33:23Z Add TO Golang Postinstall setting ports commit 9e29ae9e3a28c45fefe0943b1e054b4c5786a663 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-22T01:46:33Z Add TO Golang setcap to RPM to allow low port commit ad8020c6714f780737c4d539b24c0defe3915567 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-23T18:12:11Z Add TO Golang Perl config parser commit e6d578cc484ae571394eaa0929b6eba450e6e22b Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-23T20:53:08Z Add TO Golang old perl config reading commit 8255482375519ca8c5c9c4fe45def6b21915648b Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-25T03:53:33Z Fix TO Golang access.log text commit 68978007b28e31e029ee2e34f773f8cf269ffc33 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-25T16:38:14Z Fix TO Golang perlcfg test commit 2d765e386f3381187e3139ff9c71d5022ee48539 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-25T16:44:06Z Change TO Golang 'no_auth' cfg key to 'insecure' commit 2c2eae268859ee00f0637cc082658b6c2461b86e Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-26T04:50:43Z Change TO Golang config parse to return all errs commit 9f0007fca367930f591dd14a3441e2b84a145587 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-26T23:27:31Z Fix TO GOlang swapped log bytes and time commit f602950b1f89182afe01137110e26151434044dd Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-26T23:49:38Z Add TO Golang access log for proxied requests commit fe20f354d2b4514c3b10549aee5336e82efcf9a0 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-26T23:51:16Z Add TO Golang route for path without .json commit 4aa0048c0a5db0a2ec8f4e6a1ca53630dc9b162d Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-26T23:57:40Z Add TO Golang CORS headers commit 056bc2726f06986adb5ef693fd1b7f765a32ccbf Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-27T00:20:07Z Add TO Golang documentation commit 43046f1a5107b1cd4a5add74b667f529b203a3e4 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-27T00:28:25Z Add TO Golang logrotate commit 6b053a220f95d53cbfbf56e77b528c6ad3591a44 Author: Robert Butts <robert.o.bu...@gmail.com>
[GitHub] incubator-trafficcontrol pull request #772: Fix Traffic Monitor to startup w...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/772#discussion_r131534318
--- Diff: traffic_monitor_golang/traffic_monitor/threadsafe/polledcaches.go
---
@@ -103,11 +116,22 @@ func copyCaches(a map[enum.CacheName]struct{})
map[enum.CacheName]struct{} {
return b
}
+func copyCachesTime(a map[enum.CacheName]time.Time)
map[enum.CacheName]time.Time {
+ b := map[enum.CacheName]time.Time{}
+ for k, v := range a {
+ b[k] = v
+ }
+ return b
+}
+
+const PolledBytesPerSecTimeout = time.Second * 60
--- End diff --
Changed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #772: Fix Traffic Monitor to startup w...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/772#discussion_r131190134
--- Diff: traffic_monitor_golang/traffic_monitor/threadsafe/polledcaches.go
---
@@ -103,11 +116,22 @@ func copyCaches(a map[enum.CacheName]struct{})
map[enum.CacheName]struct{} {
return b
}
+func copyCachesTime(a map[enum.CacheName]time.Time)
map[enum.CacheName]time.Time {
+ b := map[enum.CacheName]time.Time{}
+ for k, v := range a {
+ b[k] = v
+ }
+ return b
+}
+
+const PolledBytesPerSecTimeout = time.Second * 60
--- End diff --
It makes me anxious that if we had trouble communicating with a cache which
was overloaded with traffic for 10s, we'd tell the Router it's online and has 0
bytes/second and to send it lots of traffic.
It'd take a bit more work to make it a config, that object in the code has
no access to the config right now. I'm not a big fan of adding configs for
every possible thing, though, every config option is more ops work and
training. One doesn't seem like a big deal, but they add up. This should be a
rare case; it just happens that we've allowed people to put a dysfunctional
'cache' on our CDN.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #772: Fix Traffic Monitor to startup w...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/772 Fix Traffic Monitor to startup when cache bytes don't change This fixes an issue where TM refused to start if a cache's bytes in astats never change. TM returns 503 until it's polled every cache twice, and has a BytesPerSecond calculation. The issue was, if a cache had hard-coded proc.net.dev which never changed, TM has no way to know the cache simply hasn't updated its stats, and that we won't get a different stat on the next poll. This gives TM a timeout, once it successfully polls a cache once, if the stats don't change in 60s, we start up anyway. It's not reasonable for a cache to update stats less frequently than a minute. While guessing isn't ideal, there's no good alternative. We can't know the cache simply hasn't updated its stats internally; the only option is a timeout, or never starting. Incidentally, this problem occurred in production. We had a bad cache lying to us with hard-coded astats that never updated, causing TM to never start. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol tm-fixstartupwithbadastats Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/772.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #772 commit 4fd0b6d4375f8a78254aff970c9bdb696bc56bc5 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-08-03T14:29:00Z Fix TM2 to startup when cache bytes don't change This fixes an issue where TM refused to start if a cache's bytes in astats never change. TM returns 503 until it's polled every cache twice, and has a BytesPerSecond calculation. The issue was, if a cache had hard-coded proc.net.dev which never changed, TM has no way to know the cache simply hasn't updated its stats, and that we won't get a different stat on the next poll. This gives TM a timeout, once it successfully polls a cache once, if the stats don't change in 60s, we start up anyway. It's not reasonable for a cache to update stats less frequently than a minute. While guessing isn't ideal, there's no good alternative. We can't know the cache simply hasn't updated its stats internally; the only option is a timeout, or never starting. Incidentally, this problem occurred in production. We had a bad cache lying to us with hard-coded astats that never updated, causing TM to never start. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r130890812
--- Diff: traffic_ops/install/bin/_postinstall ---
@@ -183,6 +183,22 @@ sub generateDbConf {
return \%todbconf;
--- End diff --
Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r130758948 --- Diff: docs/source/admin/traffic_ops/configuration.rst --- @@ -296,5 +296,8 @@ This is a standard kickstart formatted file that the generate ISO process uses t .. seealso:: For in-depth instructions, please see `Kickstart Installation <https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/s1-kickstart2-howuse.html>`_ +Configuring the Go Application +=== +Traffic Ops is in the process of migrating from Perl to Go, and currently runs as two applications. The Go application serves all endpoints which have been rewritten in the Go language, and transparently proxies all other requests to the old Perl application. Both applications are installed by the RPM, and both run as a single service. When the project has fully migrated to Go, the Perl application will be removed, and the RPM and service will consist solely of the Go application. - +By default, the postinstall script configures the Go application to behave and transparently serve as the old Perl Traffic Ops did in previous versions. This includes reading the old ``cdn.conf`` and ``database.conf`` config files, and logging to the old ``access.log`` location. However, if you wish to customize the Go Traffic Ops application, you can do so by running it with the ``-oldcfg=false`` argument. By default, it will then look for a config file in ``/opt/traffic_ops/conf/traffic_ops_golang.json``. The new config file location may also be customized via the ``-cfg`` flag. A sample config file is installed by the RPM at ``/opt/traffic_ops/conf/traffic_ops_golang.json``. If you wish to run the new Go Traffic Ops application as a service with a new config file, the ``-oldcfg=false`` and ``-cfg`` flags may be added to the ``start`` function in the service file, located by default at ``etc/init.d/traffic_ops``. --- End diff -- Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r130754867 --- Diff: traffic_ops/traffic_ops_golang/vendor/gopkg.in/DATA-DOG/go-sqlmock/LICENSE --- @@ -0,0 +1,28 @@ +The three clause BSD license (http://en.wikipedia.org/wiki/BSD_licenses) + --- End diff -- Done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r130709846 --- Diff: traffic_monitor_golang/traffic_monitor/config/config.go --- @@ -21,7 +21,6 @@ package config --- End diff -- The config changes have been moved into https://github.com/apache/incubator-trafficcontrol/pull/620 which has been merged. I'll rebase this to remove them and resolve the conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 I'll make the config for `MaxOpenConns`. I looked in the Go source, and Go will block if it hits that limit, rather than asynchronously returning an error. Which should be fine. I'd vote we omit `MaxIdleConns` and `ConnMaxLifetime`, until we find a need. `MaxIdleConns` is strictly less than `MaxOpenConns`, and it shouldn't hurt to leave them around, especially for a web service that continuously uses them. It's not reasonable for a SQL server to have a max connection lifetime, web services keeping connections forever is typical, so leaving that infinite should be fine too. Every additional config is more operations cost to deploy, so we should keep configs to a minimum until someone has a need. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r130630953 --- Diff: docs/source/admin/traffic_ops/configuration.rst --- @@ -296,5 +296,8 @@ This is a standard kickstart formatted file that the generate ISO process uses t .. seealso:: For in-depth instructions, please see `Kickstart Installation <https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/s1-kickstart2-howuse.html>`_ +Configuring the Go Application +=== +Traffic Ops is in the process of migrating from Perl to Go, and currently runs as two applications. The Go application serves all endpoints which have been rewritten in the Go language, and transparently proxies all other requests to the old Perl application. Both applications are installed by the RPM, and both run as a single service. When the project has fully migrated to Go, the Perl application will be removed, and the RPM and service will consist solely of the Go application. - +By default, the postinstall script configures the Go application to behave and transparently serve as the old Perl Traffic Ops did in previous versions. This includes reading the old ``cdn.conf`` and ``database.conf`` config files, and logging to the old ``access.log`` location. However, if you wish to customize the Go Traffic Ops application, you can do so by running it with the ``-oldcfg=false`` argument. By default, it will then look for a config file in ``/opt/traffic_ops/conf/traffic_ops_golang.json``. The new config file location may also be customized via the ``-cfg`` flag. A sample config file is installed by the RPM at ``/opt/traffic_ops/conf/traffic_ops_golang.json``. If you wish to run the new Go Traffic Ops application as a service with a new config file, the ``-oldcfg=false`` and ``-cfg`` flags may be added to the ``start`` function in the service file, located by default at ``etc/init.d/traffic_ops``. --- End diff -- Ah, you're right. I meant to rename the file and forgot. I'll change the documentation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #769: Fix TM2 crashing with malformed ...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/769 Fix TM2 crashing with malformed astats You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol tm2-statbug Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/769.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #769 commit c8f0d888128a8e61f1a5ddffe610ae913ca44ad0 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-31T14:10:42Z Fix TM2 crashing with malformed astats --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r130103970
--- Diff: traffic_ops/traffic_ops_golang/perlhash.go ---
@@ -0,0 +1,249 @@
+package main
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+*/
+
+import (
+ "fmt"
+ "strconv"
+ "strings"
+ "unicode"
+)
+
+func ParsePerlObj(s string) (map[string]interface{}, error) {
+ obj, _, err := getObj(s)
+ return obj, err
+}
+
+func getObj(s string) (map[string]interface{}, string, error) {
+ obj := map[string]interface{}{}
+
+ s = strings.TrimSpace(s)
+ if len(s) < 1 || s[0] != '{' {
+ return obj, "", fmt.Errorf("expected first character '{': %v",
s)
+ }
+ s = s[1:] // strip opening {
+ s = strings.TrimSpace(s)
+
+ // read top-level keys
+ for {
+ s = stripComment(s)
+ s = strings.TrimSpace(s)
+ // s = stripComment(s)
+ if len(s) > 0 && s[0] == '}' {
+ return obj, s[1:], nil
+ }
+
+ key := ""
+ key, s = getKey(s)
+
+ s = strings.TrimSpace(s)
+ if len(s) == 0 {
+ return obj, "", fmt.Errorf("malformed string after key
'%v'", key)
+ }
+
+ err := error(nil)
+ switch {
+ case s[0] == '{':
+ v := map[string]interface{}{}
+ v, s, err = getObj(s)
+ if err != nil {
+ return obj, "", fmt.Errorf("Error getting
object value after key %v: %v", key, err)
+ }
+ obj[key] = v
+ case s[0] == '\'':
+ v := ""
+ v, s, err = getStr(s)
+ if err != nil {
+ return obj, "", fmt.Errorf("Error getting
string value after key %v: %v", key, err)
+ }
+ obj[key] = v
+ case unicode.IsDigit(rune(s[0])):
--- End diff --
Eh, I care, but it'd be very expensive time-wise to fix properly. I'm sure
there are other cases (in fact, we know it doesn't support quoted keys, or
double-quoted values). But it works for the Perl config as generated by
Postinstall. And since it's only temporary until Perl goes away, I'm voting we
don't further delay Step 0 of the migration.
If we hit issues in the future, we can always go back and make the parser
more robust.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r129972669 --- Diff: traffic_ops/traffic_ops_golang/monitoring.go --- @@ -1,90 +1,45 @@ -// Licensed under the Apache License, Version 2.0 (the "License"); --- End diff -- This isn't a monitoring change, this is the `/monitoring` endpoint in Traffic Ops. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r129972542 --- Diff: traffic_monitor_golang/traffic_monitor/config/config.go --- @@ -21,7 +21,6 @@ package config --- End diff -- Because the `log` package is used by both, and the `log.InitCfg` was changed to take an interface, requiring the Monitor `Config` be given that interface's functions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r129972217
--- Diff: traffic_monitor_golang/common/log/log.go ---
@@ -95,6 +97,11 @@ func Eventf(t time.Time, format string, v
...interface{}) {
Event.Printf("%.3f %s",
float64(t.Unix())+(float64(t.Nanosecond())/1e9), fmt.Sprintf(format, v...))
}
+// EventfRaw writes to the event log with no prefix.
+func EventfRaw(format string, v ...interface{}) {
--- End diff --
No, it's using the Event logger for `access.log`. Which seemed to make
sense, accesses are events for this app.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r129972005 --- Diff: docs/source/admin/traffic_ops/configuration.rst --- @@ -296,5 +296,8 @@ This is a standard kickstart formatted file that the generate ISO process uses t .. seealso:: For in-depth instructions, please see `Kickstart Installation <https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/s1-kickstart2-howuse.html>`_ +Configuring the Go Application +=== +Traffic Ops is in the process of migrating from Perl to Go, and currently runs as two applications. The Go application serves all endpoints which have been rewritten in the Go language, and transparently proxies all other requests to the old Perl application. Both applications are installed by the RPM, and both run as a single service. When the project has fully migrated to Go, the Perl application will be removed, and the RPM and service will consist solely of the Go application. - +By default, the postinstall script configures the Go application to behave and transparently serve as the old Perl Traffic Ops did in previous versions. This includes reading the old ``cdn.conf`` and ``database.conf`` config files, and logging to the old ``access.log`` location. However, if you wish to customize the Go Traffic Ops application, you can do so by running it with the ``-oldcfg=false`` argument. By default, it will then look for a config file in ``/opt/traffic_ops/conf/traffic_ops_golang.json``. The new config file location may also be customized via the ``-cfg`` flag. A sample config file is installed by the RPM at ``/opt/traffic_ops/conf/traffic_ops_golang.json``. If you wish to run the new Go Traffic Ops application as a service with a new config file, the ``-oldcfg=false`` and ``-cfg`` flags may be added to the ``start`` function in the service file, located by default at ``etc/init.d/traffic_ops``. --- End diff -- `conf/config` seemed redundant, whereas `.json` immediately tells anyone looking at it what the format is. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #761: Add TO client DS ByServer, Regex...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/761#discussion_r129882952
--- Diff: traffic_ops/client/delivery_service_endpoints.go ---
@@ -22,6 +22,10 @@ func deliveryServicesEp() string {
return apiBase + dsPath + ".json"
}
+func deliveryServicesByServerEp(id string) string {
+ return apiBase + "/servers/" + id + dsPath + ".json"
--- End diff --
Right, I was just trying to be consistent with the surrounding functions
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #761: Add TO client DS ByServer, Regex...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/761 Add TO client DS ByServer, Regexes Adds missing CDN DomainName field You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol toclientfuncs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/761.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #761 commit 0b65316f749f0cd9634b9c19da10049164e3bdee Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-27T14:56:04Z Add TO client DS ByServer, Regexes Adds missing CDN DomainName field --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r129481493
--- Diff: traffic_ops/traffic_ops_golang/config.go ---
@@ -0,0 +1,127 @@
+package main
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "encoding/json"
+ "fmt"
+ "io/ioutil"
+ "net/url"
+
+
"github.com/apache/incubator-trafficcontrol/traffic_monitor_golang/common/log"
+)
+
+type Config struct {
+ HTTPPort string `json:"port"`
+ DBUser string `json:"db_user"`
+ DBPass string `json:"db_pass"`
+ DBServer string `json:"db_server"`
+ DBDB string `json:"db_name"`
+ DBSSL bool `json:"db_ssl"`
+ TOSecret string `json:"to_secret"`
+ TOURLStr string `json:"to_url"`
+ TOURL *url.URL `json:"-"`
+ NoAuth bool `json:"no_auth"`
+ CertPath string `json:"cert_path"`
+ KeyPathstring `json:"key_path"`
+ LogLocationError string `json:"log_location_error"`
+ LogLocationWarning string `json:"log_location_warning"`
+ LogLocationInfostring `json:"log_location_info"`
+ LogLocationDebug string `json:"log_location_debug"`
+ LogLocationEvent string `json:"log_location_event"`
+}
+
+func (c Config) Error() log.LogLocation { return
log.LogLocation(c.LogLocationError) }
+func (c Config) Warning() log.LogLocation { return
log.LogLocation(c.LogLocationWarning) }
+func (c Config) Info() log.LogLocation{ return
log.LogLocation(c.LogLocationInfo) }
+func (c Config) Debug() log.LogLocation { return
log.LogLocation(c.LogLocationDebug) }
+func (c Config) Event() log.LogLocation { return
log.LogLocation(c.LogLocationEvent) }
+
+func LoadConfig(fileName string) (Config, error) {
+ if fileName == "" {
+ return Config{}, fmt.Errorf("no filename")
+ }
+
+ configBytes, err := ioutil.ReadFile(fileName)
+ if err != nil {
+ return Config{}, err
+ }
+
+ cfg := Config{}
+ if err := json.Unmarshal(configBytes, ); err != nil {
+ return Config{}, err
+ }
+
+ if cfg, err = ParseConfig(cfg); err != nil {
+ return Config{}, err
+ }
+
+ return cfg, nil
+}
+
+// ParseConfig validates required fields, and parses non-JSON types
+func ParseConfig(cfg Config) (Config, error) {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/729#discussion_r129080524
--- Diff: traffic_ops/traffic_ops_golang/perlconfig.go ---
@@ -0,0 +1,288 @@
+package main
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "encoding/json"
+ "fmt"
+ "io/ioutil"
+ "net/url"
+ "regexp"
+ "strconv"
+ "strings"
+
+
"github.com/apache/incubator-trafficcontrol/traffic_monitor_golang/common/log"
+)
+
+const OldAccessLogPath = "/var/log/traffic_ops/access.log"
+const NewLogPath = "/var/log/traffic_ops/traffic_ops_golang.log"
+
+func GetPerlConfigs(cdnConfPath string, dbConfPath string) (Config, error)
{
+ configBytes, err := ioutil.ReadFile(cdnConfPath)
+ if err != nil {
+ return Config{}, fmt.Errorf("reading CDN conf '%v': %v",
cdnConfPath, err)
+ }
+ dbConfBytes, err := ioutil.ReadFile(dbConfPath)
+ if err != nil {
+ return Config{}, fmt.Errorf("reading db conf '%v': %v",
dbConfPath, err)
+ }
+ return getPerlConfigsFromStrs(string(configBytes), string(dbConfBytes))
+}
+
+func getPerlConfigsFromStrs(cdnConfBytes string, dbConfBytes string)
(Config, error) {
+ cfg, err := getCDNConf(cdnConfBytes)
+ if err != nil {
+ return Config{}, fmt.Errorf("parsing CDN conf '%v': %v",
cdnConfBytes, err)
+ }
+
+ dbconf, err := getDbConf(string(dbConfBytes))
+ if err != nil {
+ return Config{}, fmt.Errorf("parsing db conf '%v': %v",
dbConfBytes, err)
+ }
+ cfg.DBUser = dbconf.User
+ cfg.DBPass = dbconf.Password
+ cfg.DBServer = dbconf.Hostname
+ cfg.DBDB = dbconf.DBName
+ cfg.DBSSL = false // TODO fix
+ if dbconf.Port != "" {
+ cfg.DBServer += ":" + dbconf.Port
+ }
+
+ cfg.LogLocationInfo = OldAccessLogPath
+ cfg.LogLocationError = NewLogPath
+ cfg.LogLocationWarning = NewLogPath
+ cfg.LogLocationEvent = NewLogPath
+ cfg.LogLocationDebug = log.LogLocationNull
+
+ return cfg, nil
+}
+
+func getCDNConf(s string) (Config, error) {
+ cfg := Config{}
+ obj, err := ParsePerlObj(s)
+ if err != nil {
+ return Config{}, fmt.Errorf("parsing Perl object: %v", err)
+ }
+
+ if cfg.HTTPPort, err = getPort(obj); err != nil {
+ return Config{}, err
+ }
+
+ if cfg.TOSecret, err = getSecret(obj); err != nil {
+ return Config{}, err
+ }
+
+ oldPort, err := getOldPort(obj)
+ if err != nil {
+ return Config{}, err
+ }
+ cfg.TOURLStr = "https://127.0.0.1:; + oldPort
+ if cfg.TOURL, err = url.Parse(cfg.TOURLStr); err != nil {
+ return Config{}, fmt.Errorf("Invalid Traffic Ops URL '%v':
err", cfg.TOURL, err)
+ }
+
+ cfg.CertPath, err = getConfigCert(obj)
+ if err != nil {
+ return Config{}, err
+ }
+
+ cfg.KeyPath, err = getConfigKey(obj)
+ if err != nil {
+ return Config{}, err
+ }
+
+ return cfg, nil
+}
+
+func getPort(obj map[string]interface{}) (string, error) {
+ portStrI, ok := obj["traffic_ops_golang_port"]
+ if !ok {
+ return "", fmt.Errorf("missing traffic_ops_golang_port key")
+ }
+ portStr, ok := portStrI.(string)
+ if !ok {
--- End diff --
It's a string just because it was easier to work with in the code, e.g. the
Go HTTP server takes a string. I can make it an `int` or `uint` if you want.
I'd rather not `uint16` though, it's unusual, and even if performance mattered
it's not any faster on a 64-bit processor.
---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 Need to add `setcap 'cap_net_bind_service=+ep' /opt/traffic_ops/traffic_ops_golang` to the RPM, and a dependency on libcap2, so the Golang service can run as a user and serve 443. See https://stackoverflow.com/a/414258/292623 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 Things to do before this can be merged: - [ ] Configurable Logging (/traffic_monitor_golang/common/log) - [ ] Tests - [ ] Documentation - [ ] Postinstall adding new high port for old TO - [x] Move to root dir for build_all.sh - [ ] Golang Perl Config parser --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 We're also going to need a Perl serialized hash parser in Go, for `cdn.conf`. It _must_ be modular, so replacing it with JSON is easy once Perl TO is dead. Tight coupling with Perl hash configs is not acceptable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 @dneuman64 I agree, I'll add docs and tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 > so then you wouldn't have to touch poinstall If we want to automatically set up the config, we're going to have to touch postinstall, if nothing else, to determine the new port to serve old-TO on. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 Putting a reminder here: Also need to make setting up the config part of Postinstall. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 If you didn't set up the config, it will fail to start. Right now, failure to start will be logged to the SystemD service log. I'll make log locations part of the config, consistent with our existing apps. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #731: [TC-192] Differentiates between goTM an...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/731 I think it's been long enough, we haven't seen any major issues with the Golang TM. What about renaming `/traffic_monitor_golang` to `/traffic_monitor`, deleting the old Java dir (which will still be in Git history, if anyone needs it), and making Golang the default build in `build_all.sh`? Should we ask on the mailing list if anyone objects to making the Golang TM the default in the next release? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 > upgrade to perform all the config changes necessary to maintain > allocate a new, high, port > Notify the user via a message during upgrade @alficles Agree, will do. It'll take me a bit though, I'm not an RPM Wizard. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 Right, it _will_ cause build changes, don't merge this until we do get consensus from the community on the mailing list (I sent an email to `users` and forwarded to `dev`, "Traffic Ops Golang Migration Proposal". --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 It should be easy to retire. Once all endpoints are in Golang, simply delete the Perl from the RPM and Service files. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #729: Traffic Ops Golang Incremental Rewrite ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/729 The reason is to make it as operationally simple as possible. That's the only reason. We can make more RPMs, microservices, etc after we've moved away from Perl. I have no objection to microservices, I'm just afraid if we try to do both at once, we'll never get it done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #729: Traffic Ops Golang Incremental R...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/729 Traffic Ops Golang Incremental Rewrite App This adds an app, which serves Traffic Ops endpoints as they're written (currently, just monitoring.json), and reverse-proxies everything else to the old Perl Traffic Ops. Includes RPM and Service files, to deploy it alongside the old TO. This can be trivially deployed alongside the old TO with 2 simple config deployment (Puppet) changes: changing the port on the old TO, and adding the small config for the new TO. **Do not** merge this without consensus on the mailing list. It modifies the RPM and _will_ affect deployment. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-gomonitoring Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/729.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #729 commit 07606b1e7ef018d2f7b0c2d68f475864c6d2e29f Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-09T03:02:56Z Add experimental Go TO proxying old Perl app commit c75817be2eb8977ffd7573b67d0e060e5f38ee03 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-09T15:22:28Z Move TO Golang microservice out of experimental commit f374350a3e767fbb95239da06258436e04a0b1e2 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-09T15:23:49Z Add traffic_ops_golang to RPM, service --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #714: Add experimental Go Traffic Ops ...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/714 Add experimental Go Traffic Ops cookie creator You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-gomonitoring Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/714.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #714 commit 94d5250725a81d3c41a3c982c10e25176cc9a875 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-06T20:06:55Z Add experimental Go Traffic Ops cookie creator --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #712: Fix Traffic Monitor 2.x MonitorC...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/712 Fix Traffic Monitor 2.x MonitorConfigPoller livelock Fixes the MonitorConfigPoller for-select to be nonblocking, making livelocks impossible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol tm2-fixpollerlivelock Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/712.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #712 commit 54b100aad152592329e7833d09dff808e9d02982 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-07-05T20:29:22Z Fix TM2 MonitorConfigPoller livelock Fixes the MonitorConfigPoller for-select to be nonblocking, making livelocks impossible. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #627: Change Traffic Ops to hide sensitive da...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/627 Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #602: Change Traffic Ops password hash...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/602#discussion_r120944998
--- Diff: traffic_ops/app/lib/Utils/Helper.pm ---
@@ -132,4 +134,18 @@ sub error {
);
}
+sub hash_pass {
+ my $pass = shift;
+ return scrypt_hash($pass, \64, 16384, 8, 1, 64);
+}
+
+sub verify_pass {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #602: Change Traffic Ops password hash...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/602#discussion_r120942549
--- Diff: traffic_ops/install/bin/_postinstall ---
@@ -28,7 +28,7 @@ use DBI;
use POSIX;
use File::Basename qw{dirname};
use File::Path qw{make_path};
-use Digest::SHA1 qw(sha1_hex);
+use Crypt::ScryptKDF qw(scrypt_hash);
--- End diff --
It is
https://github.com/apache/incubator-trafficcontrol/pull/602/files#diff-557cbcfb07ce166bc477ace4e9c9eaaaR273
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol issue #602: Change Traffic Ops password hashing to ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/602 Ah, `Helper.pm` didn't already use SHA1. Not sure how it worked for me; should be fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Looks good to me (once its rebased). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #651: Fix Traffic Ops Parameter Issues
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/651 Fix Traffic Ops Parameter Issues You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-paramsql Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/651.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #651 commit a7f684500bf9aae3c51233ff751b6eff6031369d Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-05-08T17:00:23Z Fix TO parameters commit 0606b0626015db2b896d7ce8b6a7f674b7da56e3 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-05-08T17:46:38Z Fix TO parameters --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #645: Change Traffic Ops `/update` to `priv >...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/645 @mitchell852 Done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #645: Change Traffic Ops `/update` to ...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/645 Change Traffic Ops `/update` to `priv > 10` to facilitate minimal syncds priveleges This specifically allows creating a role with privilege level between 10 and 20 (e.g. 11), for ORT/syncds, which can only access GET routes plus POST /update in order to minimize access privileges. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-syncdsminimizeprivs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/645.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #645 commit df4a45f81be5a8643bcfa4698d2b2be5e8a4bb0a Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-06-05T22:07:23Z Change TO /update to allow priv_level > 10 This specifically allows creating a role with privilege level between 10 and 20 (e.g. 11), for ORT/syncds, which can only access GET routes plus POST /update in order to minimize access privileges. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #642: Add Traffic Ops username to access log
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/642 This PR doesn't modify the log format, which was already logging `%u` (which apparently broke sometime in the past), it only makes the logger aware of the username. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #602: Change Traffic Ops password hashing to ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/602 Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #642: Add Traffic Ops username to acce...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/642 Add Traffic Ops username to access log You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-logusers Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/642.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #642 commit 4c6daac26e5e57352de02334f87545cfffdacbf2 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-06-05T16:12:54Z Add TO username to access log --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #627: Change Traffic Ops to hide sensitive da...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/627 @mitchell852 Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #627: Change Traffic Ops to hide sensi...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/627 Change Traffic Ops to hide sensitive data for non-db users Changes Traffic Ops to disallow LDAP users that don't exist in the database from seeing any sensitive information (essentially anything but graphs and general CDN-wide stats). You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol to-ldapgraphsonly Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/627.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #627 commit 7a5fd2a9fdb74d8fe54fd142526d4ceb80ecd5bf Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-05-30T21:59:39Z Change TO to hide sensitive data for non-db users Changes Traffic Ops to disallow LDAP users that don't exist in the database from seeing any sensitive information (essentially anything but graphs and general CDN-wide stats). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #620: Add Golang log nil logger suppor...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/620 Add Golang log nil logger support, for performance From profiling, the log `Format(timeFormat)` call is expensive, and incurred even for null loggers. This allows applications to set null loggers to be nil, instead of ioutil.Discard, for better performance. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol tm2-nilloggers Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/620.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #620 commit 91545c7c00c82440f6fc2bbe69a49643b5ab8558 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-05-26T16:46:48Z Add log nil logger support, for performance From profiling, the log `Format(timeFormat)` call is expensive, and incurred even for null loggers. This allows applications to set null loggers to be nil, instead of ioutil.Discard, for better performance. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #583: [Backport TC-340] Remove TS unne...
Github user rob05c closed the pull request at: https://github.com/apache/incubator-trafficcontrol/pull/583 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #602: Change Traffic Ops password hash...
GitHub user rob05c opened a pull request:
https://github.com/apache/incubator-trafficcontrol/pull/602
Change Traffic Ops password hashing to scrypt
Note this is not a security vulnerability or mitigation in itself. In the
event the database is compromised, it prevents an attacker from learning the
users' passwords.
Which is the intention of hashing the passwords in the first place; but
sha1 doesn't accomplish that. Nor does sha512, the problem isn't sha1's
brokenness, it's that fast hashes aren't designed to solve this problem. The
hash must be computationally slow ("slow" here means several milliseconds).
Scrypt is a stretching hash, and solves the problem.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rob05c/incubator-trafficcontrol
to-scryptpasses
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafficcontrol/pull/602.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #602
commit 670f86cc0a549a346a63d493b75d499e833b6f09
Author: Robert Butts <robert.o.bu...@gmail.com>
Date: 2017-05-23T17:04:18Z
Change TO password hashing to scrypt
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #583: Remove TS unnecessary vendored d...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/583 Remove TS unnecessary vendored deps (cherry picked from commit 9ce2b89c0636c285d414865a900b7b4b43bd28bb) You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol ts-removeunnecessarydeps-2x Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/583.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #583 commit 7b4427bc5949c5089eb9bdf04b6e1c0c9c79cbab Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-05-17T17:17:05Z Remove TS unnecessary vendored deps (cherry picked from commit 9ce2b89c0636c285d414865a900b7b4b43bd28bb) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #580: Remove TS unnecessary vendored d...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/580 Remove TS unnecessary vendored deps These notably have incompatible licenses and _must_ be removed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol ts-removeunnecessarydeps Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/580.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #580 commit bd4c9270728e93af55cca314ad227edcd3f8cf2d Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-05-17T17:17:05Z Remove TS unnecessary vendored deps --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #576: Fix TO Docs to specify URL/user/...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/576 Fix TO Docs to specify URL/user/pass change. Fixes https://github.com/Comcast/traffic_control/issues/127 You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol docs-ort-cdn Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/576.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #576 commit 0f4aee53b8c512643ac57337e71796767a452b0b Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-05-15T19:31:16Z Fix TO Docs to specify URL/user/pass change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #569: Fix TM2 GUI table header being d...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/569 Fix TM2 GUI table header being deleted You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol tm2-fixguiheader Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/569.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #569 commit d6827bee0a86b27c4edc04a569a83309e4719a38 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-05-11T20:09:38Z Fix TM2 GUI table header being deleted --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #567: API GW phase 0 (replaces #551, d...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/567#discussion_r116004779
--- Diff: traffic_ops/experimental/webfront/webfront.go ---
@@ -122,188 +145,324 @@ func main() {
Logger.Fatal(http.ListenAndServeTLS(":" +
strconv.Itoa(int(config.ListenPort)), "server.pem", "server.key", s))
}
-func validateToken(tokenString string) (*jwt.Token, error) {
-
- tokenString = strings.Replace(tokenString, "Bearer ", "", 1)
- token, err := jwt.ParseWithClaims(tokenString, {}, func(token
*jwt.Token) (interface{}, error) {
- if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
- return nil, fmt.Errorf("Unexpected signing method: %v",
token.Header["alg"])
- }
- return []byte(os.Args[2]), nil
- })
- return token, err
-}
-
-// NewServer constructs a Server that reads rules from file with a period
-// specified by poll.
+// NewServer constructs a Server that reads Rules from file with a period
+// specified by poll
func NewServer(file string, poll time.Duration) (*Server, error) {
s := new(Server)
if err := s.loadRules(file); err != nil {
- Logger.Fatal("Error loading rules file: ", err)
+ Logger.Fatal(fmt.Errorf("Load rules failed: %s", err))
}
+
+ // TODO(amiry) - Reload config using NOHUP signal instead of poll for
changes
go s.refreshRules(file, poll)
+
return s, nil
}
-// ServeHTTP matches the Request with a Rule and, if found, serves the
-// request with the Rule's handler. If the rule's secure field is true, it
will
-// only allow access if the request has a valid JWT bearer token.
-func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+func makeTLSConfig(config *Config) *tls.Config {
- rule := s.getRule(r)
- if rule == nil {
- Logger.Printf("%v %v No mapping in rules file!", r.Method,
r.URL.RequestURI())
- http.Error(w, "Not found", http.StatusNotFound)
- return
+ s := false
+ if config.InsecureSkipVerify == true {
+ Logger.Printf("NOTICE: Skip sertificate verification")
--- End diff --
Typo 'sertificate'
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115551147
--- Diff: traffic_ops/experimental/webfront/webfront.go ---
@@ -122,188 +145,311 @@ func main() {
Logger.Fatal(http.ListenAndServeTLS(":" +
strconv.Itoa(int(config.ListenPort)), "server.pem", "server.key", s))
}
-func validateToken(tokenString string) (*jwt.Token, error) {
-
- tokenString = strings.Replace(tokenString, "Bearer ", "", 1)
- token, err := jwt.ParseWithClaims(tokenString, {}, func(token
*jwt.Token) (interface{}, error) {
- if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
- return nil, fmt.Errorf("Unexpected signing method: %v",
token.Header["alg"])
- }
- return []byte(os.Args[2]), nil
- })
- return token, err
-}
-
-// NewServer constructs a Server that reads rules from file with a period
-// specified by poll.
+// NewServer constructs a Server that reads Rules from file with a period
+// specified by poll
func NewServer(file string, poll time.Duration) (*Server, error) {
s := new(Server)
if err := s.loadRules(file); err != nil {
- Logger.Fatal("Error loading rules file: ", err)
+ Logger.Fatal(fmt.Errorf("Load rules failed: %s", err))
}
go s.refreshRules(file, poll)
return s, nil
}
-// ServeHTTP matches the Request with a Rule and, if found, serves the
-// request with the Rule's handler. If the rule's secure field is true, it
will
-// only allow access if the request has a valid JWT bearer token.
-func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+// loadRules tests whether file has been modified since its last invocation
+// and, if so, loads the rule set from file.
+func (s *Server) loadRules(file string) error {
- rule := s.getRule(r)
- if rule == nil {
- Logger.Printf("%v %v No mapping in rules file!", r.Method,
r.URL.RequestURI())
- http.Error(w, "Not found", http.StatusNotFound)
- return
+ fi, err := os.Stat(file)
+ if err != nil {
+ return err
}
- isAuthorized := false
+ mtime := fi.ModTime()
+ if !mtime.After(s.last) && s.Rules != nil {
+ return nil // no change
+ }
- if rule.Secure {
- tokenValid := false
- token, err := validateToken(r.Header.Get("Authorization"))
+ Rules, err := parseRules(file)
+ if err != nil {
+ return err
+ }
- if err == nil {
- tokenValid = true
- } else {
- Logger.Println("Token Error:", err.Error())
+ s.mu.Lock()
+ s.last = mtime
+ s.Rules = Rules
+ s.mu.Unlock()
+ return nil
+}
+
+// refreshRules polls file periodically and refreshes the Server's rule set
+// if the file has been modified.
+func (s *Server) refreshRules(file string, poll time.Duration) {
+ for {
+ if err := s.loadRules(file); err != nil {
+ Logger.Printf("Refresh rules failed: %s", err)
}
+ time.Sleep(poll)
+ }
+}
- if !tokenValid {
- Logger.Printf("%v %v Valid token required, but none
found!", r.Method, r.URL.RequestURI())
- w.WriteHeader(http.StatusForbidden)
- return
+// parseRules reads rule definitions from file, constructs the rule
handlers,
+// and returns the resultant rules.
+func parseRules(file string) ([]*FwdRule, error) {
+
+ f, err := os.Open(file)
+ if err != nil {
+ return nil, err
+ }
+ defer f.Close()
+
+ Logger.Printf("Loading rules file: %s", file)
+
+ var rules []*FwdRule
+ if err := json.NewDecoder(f).Decode(); err != nil {
+ return nil, err
+ }
+
+ for _, r := range rules {
+
+ if r.Auth {
+ r.routes, err = parseRoutes(r.RoutesFile)
+ if err != nil {
+ Logger.Printf("Skip rule %s ERROR: %s", r.Path,
err)
+ continue
+ }
}
- claims, ok := token.Claims.(*Claims)
- if !ok {
- Logger.Printf("%v %v Valid token found, but cannot
parse claims!", r.Method, r.URL.RequestURI())
- w.WriteHeader(ht
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115150493 --- Diff: traffic_ops/experimental/auth/README.md --- @@ -1,6 +1,13 @@ A simple authentication server written in go that authenticates user agains the `tm_user` table and returns a jwt representing the user, incl. its API access capabilities, derived from the user's role. + Legacy TO support + +Legacy TO authorization code requires any API call to pass a mojolicios access token in its access control headers. +Untill this code is deprecated, the Auth server and the API GW handle legacy authorization in hte following way: +Upon every sucessful login the auth server performs additional login against legacy TO (mojolicious app) and recieves a lagacy TO authentication token. --- End diff -- Typo, "legacy" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115150424 --- Diff: traffic_ops/experimental/auth/README.md --- @@ -1,6 +1,13 @@ A simple authentication server written in go that authenticates user agains the `tm_user` table and returns a jwt representing the user, incl. its API access capabilities, derived from the user's role. + Legacy TO support + +Legacy TO authorization code requires any API call to pass a mojolicios access token in its access control headers. +Untill this code is deprecated, the Auth server and the API GW handle legacy authorization in hte following way: --- End diff -- Typo, "in the" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request: https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115150411 --- Diff: traffic_ops/experimental/auth/README.md --- @@ -1,6 +1,13 @@ A simple authentication server written in go that authenticates user agains the `tm_user` table and returns a jwt representing the user, incl. its API access capabilities, derived from the user's role. + Legacy TO support + +Legacy TO authorization code requires any API call to pass a mojolicios access token in its access control headers. --- End diff -- Typo, "mojolicious" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149357
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149292
--- Diff: traffic_ops/experimental/webfront/webfront.go ---
@@ -122,188 +145,311 @@ func main() {
Logger.Fatal(http.ListenAndServeTLS(":" +
strconv.Itoa(int(config.ListenPort)), "server.pem", "server.key", s))
}
-func validateToken(tokenString string) (*jwt.Token, error) {
-
- tokenString = strings.Replace(tokenString, "Bearer ", "", 1)
- token, err := jwt.ParseWithClaims(tokenString, {}, func(token
*jwt.Token) (interface{}, error) {
- if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
- return nil, fmt.Errorf("Unexpected signing method: %v",
token.Header["alg"])
- }
- return []byte(os.Args[2]), nil
- })
- return token, err
-}
-
-// NewServer constructs a Server that reads rules from file with a period
-// specified by poll.
+// NewServer constructs a Server that reads Rules from file with a period
+// specified by poll
func NewServer(file string, poll time.Duration) (*Server, error) {
s := new(Server)
if err := s.loadRules(file); err != nil {
- Logger.Fatal("Error loading rules file: ", err)
+ Logger.Fatal(fmt.Errorf("Load rules failed: %s", err))
}
go s.refreshRules(file, poll)
return s, nil
}
-// ServeHTTP matches the Request with a Rule and, if found, serves the
-// request with the Rule's handler. If the rule's secure field is true, it
will
-// only allow access if the request has a valid JWT bearer token.
-func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+// loadRules tests whether file has been modified since its last invocation
+// and, if so, loads the rule set from file.
+func (s *Server) loadRules(file string) error {
- rule := s.getRule(r)
- if rule == nil {
- Logger.Printf("%v %v No mapping in rules file!", r.Method,
r.URL.RequestURI())
- http.Error(w, "Not found", http.StatusNotFound)
- return
+ fi, err := os.Stat(file)
+ if err != nil {
+ return err
}
- isAuthorized := false
+ mtime := fi.ModTime()
+ if !mtime.After(s.last) && s.Rules != nil {
+ return nil // no change
+ }
- if rule.Secure {
- tokenValid := false
- token, err := validateToken(r.Header.Get("Authorization"))
+ Rules, err := parseRules(file)
+ if err != nil {
+ return err
+ }
- if err == nil {
- tokenValid = true
- } else {
- Logger.Println("Token Error:", err.Error())
+ s.mu.Lock()
+ s.last = mtime
+ s.Rules = Rules
+ s.mu.Unlock()
+ return nil
+}
+
+// refreshRules polls file periodically and refreshes the Server's rule set
+// if the file has been modified.
+func (s *Server) refreshRules(file string, poll time.Duration) {
+ for {
+ if err := s.loadRules(file); err != nil {
+ Logger.Printf("Refresh rules failed: %s", err)
}
+ time.Sleep(poll)
+ }
+}
- if !tokenValid {
- Logger.Printf("%v %v Valid token required, but none
found!", r.Method, r.URL.RequestURI())
- w.WriteHeader(http.StatusForbidden)
- return
+// parseRules reads rule definitions from file, constructs the rule
handlers,
+// and returns the resultant rules.
+func parseRules(file string) ([]*FwdRule, error) {
+
+ f, err := os.Open(file)
+ if err != nil {
+ return nil, err
+ }
+ defer f.Close()
+
+ Logger.Printf("Loading rules file: %s", file)
+
+ var rules []*FwdRule
+ if err := json.NewDecoder(f).Decode(); err != nil {
+ return nil, err
+ }
+
+ for _, r := range rules {
+
+ if r.Auth {
+ r.routes, err = parseRoutes(r.RoutesFile)
+ if err != nil {
+ Logger.Printf("Skip rule %s ERROR: %s", r.Path,
err)
+ continue
+ }
}
- claims, ok := token.Claims.(*Claims)
- if !ok {
- Logger.Printf("%v %v Valid token found, but cannot
parse claims!", r.Method, r.URL.RequestURI())
- w.WriteHeader(ht
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149354
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149766
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149332
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149697
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149913
--- Diff: traffic_ops/experimental/webfront/webfront.go ---
@@ -122,188 +145,311 @@ func main() {
Logger.Fatal(http.ListenAndServeTLS(":" +
strconv.Itoa(int(config.ListenPort)), "server.pem", "server.key", s))
}
-func validateToken(tokenString string) (*jwt.Token, error) {
-
- tokenString = strings.Replace(tokenString, "Bearer ", "", 1)
- token, err := jwt.ParseWithClaims(tokenString, {}, func(token
*jwt.Token) (interface{}, error) {
- if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
- return nil, fmt.Errorf("Unexpected signing method: %v",
token.Header["alg"])
- }
- return []byte(os.Args[2]), nil
- })
- return token, err
-}
-
-// NewServer constructs a Server that reads rules from file with a period
-// specified by poll.
+// NewServer constructs a Server that reads Rules from file with a period
+// specified by poll
func NewServer(file string, poll time.Duration) (*Server, error) {
--- End diff --
Polling to reload configs is unpredictable, and prone to error if a user is
in the middle of editing a config. The standard method is via NOHUP signal,
which most service management systems will send on `service reload`.
This is ok for /experimental, but you should plan on changing it to Nohup
before it's moved out. See
https://github.com/apache/incubator-trafficcontrol/blob/8eda1c7f510c1a6504ddf2ef125ba14aaccd523a/traffic_stats/traffic_stats.go#L137
for an example in Go.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115150002
--- Diff: traffic_ops/experimental/webfront/webfront.go ---
@@ -122,188 +145,311 @@ func main() {
Logger.Fatal(http.ListenAndServeTLS(":" +
strconv.Itoa(int(config.ListenPort)), "server.pem", "server.key", s))
}
-func validateToken(tokenString string) (*jwt.Token, error) {
-
- tokenString = strings.Replace(tokenString, "Bearer ", "", 1)
- token, err := jwt.ParseWithClaims(tokenString, {}, func(token
*jwt.Token) (interface{}, error) {
- if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
- return nil, fmt.Errorf("Unexpected signing method: %v",
token.Header["alg"])
- }
- return []byte(os.Args[2]), nil
- })
- return token, err
-}
-
-// NewServer constructs a Server that reads rules from file with a period
-// specified by poll.
+// NewServer constructs a Server that reads Rules from file with a period
+// specified by poll
func NewServer(file string, poll time.Duration) (*Server, error) {
s := new(Server)
if err := s.loadRules(file); err != nil {
- Logger.Fatal("Error loading rules file: ", err)
+ Logger.Fatal(fmt.Errorf("Load rules failed: %s", err))
}
go s.refreshRules(file, poll)
return s, nil
}
-// ServeHTTP matches the Request with a Rule and, if found, serves the
-// request with the Rule's handler. If the rule's secure field is true, it
will
-// only allow access if the request has a valid JWT bearer token.
-func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+// loadRules tests whether file has been modified since its last invocation
+// and, if so, loads the rule set from file.
+func (s *Server) loadRules(file string) error {
- rule := s.getRule(r)
- if rule == nil {
- Logger.Printf("%v %v No mapping in rules file!", r.Method,
r.URL.RequestURI())
- http.Error(w, "Not found", http.StatusNotFound)
- return
+ fi, err := os.Stat(file)
+ if err != nil {
+ return err
}
- isAuthorized := false
+ mtime := fi.ModTime()
+ if !mtime.After(s.last) && s.Rules != nil {
+ return nil // no change
+ }
- if rule.Secure {
- tokenValid := false
- token, err := validateToken(r.Header.Get("Authorization"))
+ Rules, err := parseRules(file)
+ if err != nil {
+ return err
+ }
- if err == nil {
- tokenValid = true
- } else {
- Logger.Println("Token Error:", err.Error())
+ s.mu.Lock()
+ s.last = mtime
+ s.Rules = Rules
+ s.mu.Unlock()
+ return nil
+}
+
+// refreshRules polls file periodically and refreshes the Server's rule set
+// if the file has been modified.
+func (s *Server) refreshRules(file string, poll time.Duration) {
+ for {
+ if err := s.loadRules(file); err != nil {
+ Logger.Printf("Refresh rules failed: %s", err)
}
+ time.Sleep(poll)
+ }
+}
- if !tokenValid {
- Logger.Printf("%v %v Valid token required, but none
found!", r.Method, r.URL.RequestURI())
- w.WriteHeader(http.StatusForbidden)
- return
+// parseRules reads rule definitions from file, constructs the rule
handlers,
+// and returns the resultant rules.
+func parseRules(file string) ([]*FwdRule, error) {
+
+ f, err := os.Open(file)
+ if err != nil {
+ return nil, err
+ }
+ defer f.Close()
+
+ Logger.Printf("Loading rules file: %s", file)
+
+ var rules []*FwdRule
+ if err := json.NewDecoder(f).Decode(); err != nil {
+ return nil, err
+ }
+
+ for _, r := range rules {
+
+ if r.Auth {
+ r.routes, err = parseRoutes(r.RoutesFile)
+ if err != nil {
+ Logger.Printf("Skip rule %s ERROR: %s", r.Path,
err)
+ continue
+ }
}
- claims, ok := token.Claims.(*Claims)
- if !ok {
- Logger.Printf("%v %v Valid token found, but cannot
parse claims!", r.Method, r.URL.RequestURI())
- w.WriteHeader(ht
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149679
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115148956
--- Diff: traffic_ops/experimental/webfront/webfront.go ---
@@ -122,188 +145,311 @@ func main() {
Logger.Fatal(http.ListenAndServeTLS(":" +
strconv.Itoa(int(config.ListenPort)), "server.pem", "server.key", s))
}
-func validateToken(tokenString string) (*jwt.Token, error) {
-
- tokenString = strings.Replace(tokenString, "Bearer ", "", 1)
- token, err := jwt.ParseWithClaims(tokenString, {}, func(token
*jwt.Token) (interface{}, error) {
- if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
- return nil, fmt.Errorf("Unexpected signing method: %v",
token.Header["alg"])
- }
- return []byte(os.Args[2]), nil
- })
- return token, err
-}
-
-// NewServer constructs a Server that reads rules from file with a period
-// specified by poll.
+// NewServer constructs a Server that reads Rules from file with a period
+// specified by poll
func NewServer(file string, poll time.Duration) (*Server, error) {
s := new(Server)
if err := s.loadRules(file); err != nil {
- Logger.Fatal("Error loading rules file: ", err)
+ Logger.Fatal(fmt.Errorf("Load rules failed: %s", err))
}
go s.refreshRules(file, poll)
return s, nil
}
-// ServeHTTP matches the Request with a Rule and, if found, serves the
-// request with the Rule's handler. If the rule's secure field is true, it
will
-// only allow access if the request has a valid JWT bearer token.
-func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+// loadRules tests whether file has been modified since its last invocation
+// and, if so, loads the rule set from file.
+func (s *Server) loadRules(file string) error {
- rule := s.getRule(r)
- if rule == nil {
- Logger.Printf("%v %v No mapping in rules file!", r.Method,
r.URL.RequestURI())
- http.Error(w, "Not found", http.StatusNotFound)
- return
+ fi, err := os.Stat(file)
+ if err != nil {
+ return err
}
- isAuthorized := false
+ mtime := fi.ModTime()
+ if !mtime.After(s.last) && s.Rules != nil {
+ return nil // no change
+ }
- if rule.Secure {
- tokenValid := false
- token, err := validateToken(r.Header.Get("Authorization"))
+ Rules, err := parseRules(file)
+ if err != nil {
+ return err
+ }
- if err == nil {
- tokenValid = true
- } else {
- Logger.Println("Token Error:", err.Error())
+ s.mu.Lock()
+ s.last = mtime
+ s.Rules = Rules
+ s.mu.Unlock()
+ return nil
+}
+
+// refreshRules polls file periodically and refreshes the Server's rule set
+// if the file has been modified.
+func (s *Server) refreshRules(file string, poll time.Duration) {
+ for {
+ if err := s.loadRules(file); err != nil {
+ Logger.Printf("Refresh rules failed: %s", err)
}
+ time.Sleep(poll)
+ }
+}
- if !tokenValid {
- Logger.Printf("%v %v Valid token required, but none
found!", r.Method, r.URL.RequestURI())
- w.WriteHeader(http.StatusForbidden)
- return
+// parseRules reads rule definitions from file, constructs the rule
handlers,
+// and returns the resultant rules.
+func parseRules(file string) ([]*FwdRule, error) {
+
+ f, err := os.Open(file)
+ if err != nil {
+ return nil, err
+ }
+ defer f.Close()
+
+ Logger.Printf("Loading rules file: %s", file)
+
+ var rules []*FwdRule
+ if err := json.NewDecoder(f).Decode(); err != nil {
+ return nil, err
+ }
+
+ for _, r := range rules {
+
+ if r.Auth {
+ r.routes, err = parseRoutes(r.RoutesFile)
+ if err != nil {
+ Logger.Printf("Skip rule %s ERROR: %s", r.Path,
err)
+ continue
+ }
}
- claims, ok := token.Claims.(*Claims)
- if !ok {
- Logger.Printf("%v %v Valid token found, but cannot
parse claims!", r.Method, r.URL.RequestURI())
- w.WriteHeader(ht
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149662
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149335
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149323
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149338
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149659
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149976
--- Diff: traffic_ops/experimental/webfront/webfront.go ---
@@ -122,188 +145,311 @@ func main() {
Logger.Fatal(http.ListenAndServeTLS(":" +
strconv.Itoa(int(config.ListenPort)), "server.pem", "server.key", s))
}
-func validateToken(tokenString string) (*jwt.Token, error) {
-
- tokenString = strings.Replace(tokenString, "Bearer ", "", 1)
- token, err := jwt.ParseWithClaims(tokenString, {}, func(token
*jwt.Token) (interface{}, error) {
- if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
- return nil, fmt.Errorf("Unexpected signing method: %v",
token.Header["alg"])
- }
- return []byte(os.Args[2]), nil
- })
- return token, err
-}
-
-// NewServer constructs a Server that reads rules from file with a period
-// specified by poll.
+// NewServer constructs a Server that reads Rules from file with a period
+// specified by poll
func NewServer(file string, poll time.Duration) (*Server, error) {
s := new(Server)
if err := s.loadRules(file); err != nil {
- Logger.Fatal("Error loading rules file: ", err)
+ Logger.Fatal(fmt.Errorf("Load rules failed: %s", err))
}
go s.refreshRules(file, poll)
return s, nil
}
-// ServeHTTP matches the Request with a Rule and, if found, serves the
-// request with the Rule's handler. If the rule's secure field is true, it
will
-// only allow access if the request has a valid JWT bearer token.
-func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+// loadRules tests whether file has been modified since its last invocation
+// and, if so, loads the rule set from file.
+func (s *Server) loadRules(file string) error {
- rule := s.getRule(r)
- if rule == nil {
- Logger.Printf("%v %v No mapping in rules file!", r.Method,
r.URL.RequestURI())
- http.Error(w, "Not found", http.StatusNotFound)
- return
+ fi, err := os.Stat(file)
+ if err != nil {
+ return err
}
- isAuthorized := false
+ mtime := fi.ModTime()
+ if !mtime.After(s.last) && s.Rules != nil {
+ return nil // no change
+ }
- if rule.Secure {
- tokenValid := false
- token, err := validateToken(r.Header.Get("Authorization"))
+ Rules, err := parseRules(file)
+ if err != nil {
+ return err
+ }
- if err == nil {
- tokenValid = true
- } else {
- Logger.Println("Token Error:", err.Error())
+ s.mu.Lock()
+ s.last = mtime
+ s.Rules = Rules
+ s.mu.Unlock()
+ return nil
+}
+
+// refreshRules polls file periodically and refreshes the Server's rule set
+// if the file has been modified.
+func (s *Server) refreshRules(file string, poll time.Duration) {
+ for {
+ if err := s.loadRules(file); err != nil {
+ Logger.Printf("Refresh rules failed: %s", err)
}
+ time.Sleep(poll)
+ }
+}
- if !tokenValid {
- Logger.Printf("%v %v Valid token required, but none
found!", r.Method, r.URL.RequestURI())
- w.WriteHeader(http.StatusForbidden)
- return
+// parseRules reads rule definitions from file, constructs the rule
handlers,
+// and returns the resultant rules.
+func parseRules(file string) ([]*FwdRule, error) {
+
+ f, err := os.Open(file)
+ if err != nil {
+ return nil, err
+ }
+ defer f.Close()
+
+ Logger.Printf("Loading rules file: %s", file)
+
+ var rules []*FwdRule
+ if err := json.NewDecoder(f).Decode(); err != nil {
+ return nil, err
+ }
+
+ for _, r := range rules {
+
+ if r.Auth {
+ r.routes, err = parseRoutes(r.RoutesFile)
+ if err != nil {
+ Logger.Printf("Skip rule %s ERROR: %s", r.Path,
err)
+ continue
+ }
}
- claims, ok := token.Claims.(*Claims)
- if !ok {
- Logger.Printf("%v %v Valid token found, but cannot
parse claims!", r.Method, r.URL.RequestURI())
- w.WriteHeader(ht
[GitHub] incubator-trafficcontrol pull request #551: API GW phase 0 (depends on PR #5...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/551#discussion_r115149729
--- Diff: traffic_ops/experimental/auth/auth.go ---
@@ -132,83 +150,197 @@ func InitializeDatabase(username, password, dbname,
server string, port uint) (*
return db, nil
}
-func handler(w http.ResponseWriter, r *http.Request) {
+func LegacyTOLogin(login Login, legacyLoginURL string, w
http.ResponseWriter) (*http.Response, error) {
- Logger.Println(r.Method, r.URL.Scheme, r.Host, r.URL.RequestURI())
+ // TODO(amiry) - Legacy token expiration should be longer than JWT
expiration
- if r.Method == "POST" {
- var login Login
- tmUserlist := []TmUser{}
- body, err := ioutil.ReadAll(r.Body)
- if err != nil {
- Logger.Println("Error reading body: ", err.Error())
- http.Error(w, "Error reading body: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- err = json.Unmarshal(body, )
- if err != nil {
- Logger.Println("Invalid JSON: ", err.Error())
- http.Error(w, "Invalid JSON: "+err.Error(),
http.StatusBadRequest)
- return
- }
-
- stmt, err := db.PrepareNamed("SELECT role,local_passwd FROM
tm_user WHERE username=:username")
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
-
- err = stmt.Select(, login)
- if err != nil {
- Logger.Println("Database error: ", err.Error())
- http.Error(w, "Database error: "+err.Error(),
http.StatusInternalServerError)
- return
- }
+ legacyLogin := LegacyLogin{ login.Username, login.Password }
- hasher := sha1.New()
- hasher.Write([]byte(login.Password))
- hashedPassword := fmt.Sprintf("%x", hasher.Sum(nil))
+ body, err := json.Marshal(legacyLogin)
+if err != nil {
+ Logger.Println("JSON marshal error: ", err.Error())
+return nil, err
+}
- if len(tmUserlist) == 0 || tmUserlist[0].Password !=
string(hashedPassword) {
- Logger.Printf("Invalid username/password, username %s",
login.Username)
- http.Error(w, "Invalid username/password",
http.StatusUnauthorized)
- return
- }
-
- Logger.Printf("User %s authenticated", login.Username)
-
- claims := Claims {
- []string{"read-ds", "write-ds", "read-cg"}, // TODO(amiry)
- Adding hardcoded capabilities as a POC.
-
// Need to read from TO role tables when tables are
ready
- jwt.StandardClaims {
- Subject: login.Username,
- ExpiresAt: time.Now().Add(time.Hour * 24).Unix(), //
TODO(amiry) - We will need to use shorter expiration,
-
// and use refresh tokens to extend
access
- },
- }
-
- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+ req, err := http.NewRequest("POST", legacyLoginURL,
bytes.NewBuffer(body))
+ client := {}
+resp, err := client.Do(req)
+ if err != nil {
+ Logger.Println("Legacy Login error: ", err.Error(), " Legacy
URL: ", legacyLoginURL)
+ return nil, err;
+ }
- tokenString, err := token.SignedString([]byte(os.Args[2]))
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
- return
- }
+ return resp, err
+}
- js, err := json.Marshal(TokenResponse{Token: tokenString})
- if err != nil {
- Logger.Println(err.Error())
- http.Error(w, err.Error(),
http.StatusInternalServerError)
+func makeHandler(config *Config) (func(http.ResponseWriter,
*http.Request), error) {
+
+ return func (w http.Respon
[GitHub] incubator-trafficcontrol pull request #516: Add Traffic Monitor 2.0 Paramete...
GitHub user rob05c opened a pull request: https://github.com/apache/incubator-trafficcontrol/pull/516 Add Traffic Monitor 2.0 Parameter to disable TCP KeepAlive in Polls You can merge this pull request into a Git repository by running: $ git pull https://github.com/rob05c/incubator-trafficcontrol tm2-keepalive Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafficcontrol/pull/516.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #516 commit 03112fabd9715330ac549d2b97c570870a8ff674 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-04-24T20:04:17Z Add TM2 polling.keepalive parameter Adds 'health.', 'stat.', 'peer.polling.keepalive' to disable TCP KeepAlive for certain polls. Defaults to true, unless a parameter exists which starts with 'f' or 'F'. commit 2ac111bd09855e8ba6dc785d1c506f7a7f506316 Author: Robert Butts <robert.o.bu...@gmail.com> Date: 2017-04-25T17:01:42Z Fix TM2 tmcheck not closing request bodies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #415: Add Traffic Monitor 2.0 Log Loca...
Github user rob05c closed the pull request at: https://github.com/apache/incubator-trafficcontrol/pull/415 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #425: Add Traffic Monitor 2.0 HTTP gzip suppo...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/425 As far as the Wrapper comments go: a lot of @alficles objections are difficult to get right, or impossible to implement, with the current Wrap helpers The Wrap helpers are are now more duplicate than reasonable. Go's type system isn't powerful enough for these wrappers to work well, especially as the handling complexity grows. They're increasingly difficult to write and read. I've thought for a while now that they should be refactored, probably into stock `http.HandlerFunc`s. Using `HandlerFunc` would have some duplicate logic, but probably not much more than is already there, and would be far easier to understand. I'd rather wait to do that refactor in its own PR, than here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol pull request #425: Add Traffic Monitor 2.0 HTTP gzi...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/425#discussion_r111809861
--- Diff: traffic_monitor_golang/traffic_monitor/datareq/datareq.go ---
@@ -235,3 +276,39 @@ func addTrailingSlashEndpoints(dispatchMap
map[string]http.HandlerFunc) map[stri
}
return dispatchMap
}
+
+func acceptsGzip(r *http.Request) bool {
+ encodingHeaders := r.Header["Accept-Encoding"] // headers are
case-insensitive, but Go promises to Canonical-Case requests
+ for _, encodingHeader := range encodingHeaders {
+ encodingHeader := strings.Replace(encodingHeader, " ", "", -1)
--- End diff --
Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #425: Add Traffic Monitor 2.0 HTTP gzi...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/425#discussion_r111809687
--- Diff: traffic_monitor_golang/traffic_monitor/datareq/datareq.go ---
@@ -235,3 +276,39 @@ func addTrailingSlashEndpoints(dispatchMap
map[string]http.HandlerFunc) map[stri
}
return dispatchMap
}
+
+func acceptsGzip(r *http.Request) bool {
+ encodingHeaders := r.Header["Accept-Encoding"] // headers are
case-insensitive, but Go promises to Canonical-Case requests
+ for _, encodingHeader := range encodingHeaders {
+ encodingHeader := strings.Replace(encodingHeader, " ", "", -1)
+ encodings := strings.Split(encodingHeader, ",")
+ for _, encoding := range encodings {
+ if strings.ToLower(encoding) == "gzip" { // encoding is
case-insensitive, per the RFC
+ return true
+ }
+ }
+ }
+ return false
+}
+
+// gzipIfAccepts gzips the given bytes, writes a `Content-Encoding: gzip`
header to the given writer, and returns the gzipped bytes, if the Request
supports GZip (has an Accept-Encoding header). Else, returns the bytes
unmodified. Note the given bytes are NOT written to the given writer. It is
assumed the bytes may need to pass thru other middleware before being written.
+func gzipIfAccepts(r *http.Request, w http.ResponseWriter, b []byte)
([]byte, error) {
+ // TODO this could be made more efficient by wrapping ResponseWriter
with the GzipWriter, and letting callers writer directly to it - but then we'd
have to deal with Closing the gzip.Writer.
+ if len(b) == 0 || !acceptsGzip(r) {
+ return b, nil
+ }
+ w.Header().Set("Content-Encoding", "gzip")
+
+ buf := bytes.Buffer{}
+ zw := gzip.NewWriter()
+
+ if _, err := zw.Write(b); err != nil {
+ return nil, fmt.Errorf("gzipping bytes: %v")
+ }
+
+ if err := zw.Close(); err != nil {
+ return nil, fmt.Errorf("closing gzip writer: %v")
+ }
+
+ return buf.Bytes(), nil
--- End diff --
As you say, the compressed one will be smaller 99% of the time, and when it
is, performance almost certainly doesn't matter. IMO the extra code for that
logic isn't worth the maintenance cost.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafficcontrol pull request #425: Add Traffic Monitor 2.0 HTTP gzi...
Github user rob05c commented on a diff in the pull request:
https://github.com/apache/incubator-trafficcontrol/pull/425#discussion_r111807742
--- Diff: traffic_monitor_golang/traffic_monitor/datareq/datareq.go ---
@@ -157,16 +159,34 @@ func WrapErrCode(errorCount threadsafe.Uint, reqPath
string, body []byte, err er
// WrapBytes takes a function which cannot error and returns only bytes,
and wraps it as a http.HandlerFunc. The errContext is logged if the write
fails, and should be enough information to trace the problem (function name,
endpoint, request parameters, etc).
func WrapBytes(f func() []byte, contentType string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
+ bytes := f()
+ bytes, err := gzipIfAccepts(r, w, bytes)
+ if err != nil {
+ log.Errorf("gzipping request '%v': %v\n",
r.URL.EscapedPath(), err)
+ code := http.StatusInternalServerError
+ w.WriteHeader(code)
+ if _, err := w.Write([]byte(http.StatusText(code)));
err != nil {
--- End diff --
IMO idiomatic naming is better in this case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
