[GitHub] asfgit commented on issue #2281: Increase hostname and domain name length limit in Traffic Portal
asfgit commented on issue #2281: Increase hostname and domain name length limit in Traffic Portal URL: https://github.com/apache/incubator-trafficcontrol/pull/2281#issuecomment-389681832 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1587/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rawlinp commented on issue #2279: Increase server hostname max length
rawlinp commented on issue #2279: Increase server hostname max length URL: https://github.com/apache/incubator-trafficcontrol/issues/2279#issuecomment-389679003 Sure, just opened #2281 to increase both limits in TP to 100. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rawlinp opened a new pull request #2281: Increase hostname and domain name length limit in Traffic Portal
rawlinp opened a new pull request #2281: Increase hostname and domain name length limit in Traffic Portal URL: https://github.com/apache/incubator-trafficcontrol/pull/2281 This changes is from 45 to 100. The limit itself could probably be removed altogether since the DB type is `text`, but it's probably best practice to have some kind of artificial limit in the UI anyways. Fixes #2279 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2280: Fix TO CRconfig changelog
asfgit commented on issue #2280: Fix TO CRconfig changelog URL: https://github.com/apache/incubator-trafficcontrol/pull/2280#issuecomment-389676041 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1586/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 commented on issue #2279: Increase server hostname max length
mitchell852 commented on issue #2279: Increase server hostname max length URL: https://github.com/apache/incubator-trafficcontrol/issues/2279#issuecomment-389673406 there's probably a maxLength attribute in TP. you want to remove that @rawlinp ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] elsloo closed pull request #2280: Fix TO CRconfig changelog
elsloo closed pull request #2280: Fix TO CRconfig changelog URL: https://github.com/apache/incubator-trafficcontrol/pull/2280 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/traffic_ops/traffic_ops_golang/api/change_log.go b/traffic_ops/traffic_ops_golang/api/change_log.go index 17c54a784..f2fb18f5c 100644 --- a/traffic_ops/traffic_ops_golang/api/change_log.go +++ b/traffic_ops/traffic_ops_golang/api/change_log.go @@ -20,11 +20,13 @@ package api */ import ( + "database/sql" "fmt" "github.com/apache/incubator-trafficcontrol/lib/go-log" "github.com/apache/incubator-trafficcontrol/lib/go-tc" "github.com/apache/incubator-trafficcontrol/traffic_ops/traffic_ops_golang/auth" + "github.com/jmoiron/sqlx" ) @@ -76,3 +78,10 @@ func CreateChangeLog(level string, action string, i Identifier, user auth.Curren } return nil } + +func CreateChangeLogRaw(level string, message string, user auth.CurrentUser, db *sql.DB) error { + if _, err := db.Exec(`INSERT INTO log (level, message, tm_user) VALUES ($1, $2, $3)`, level, message, user.ID); err != nil { + return fmt.Errorf("inserting change log level '%v' message '%v' user '%v': %v", level, message, user.ID, err) + } + return nil +} diff --git a/traffic_ops/traffic_ops_golang/crconfig/handler.go b/traffic_ops/traffic_ops_golang/crconfig/handler.go index 0c4453c93..6822b60c9 100644 --- a/traffic_ops/traffic_ops_golang/crconfig/handler.go +++ b/traffic_ops/traffic_ops_golang/crconfig/handler.go @@ -189,9 +189,13 @@ func SnapshotHandler(db *sqlx.DB, cfg config.Config) http.HandlerFunc { } if err := Snapshot(db.DB, crConfig); err != nil { + log.Errorln(r.RemoteAddr + " snaphsotting CRConfig: " + err.Error()) handleErrs(http.StatusInternalServerError, err) return } + if err := api.CreateChangeLogRaw(api.ApiChange, "Snapshot of CRConfig performed for "+cdn, *user, db.DB); err != nil { + log.Errorln("creating CRConfig snapshot changelog: " + err.Error()) + } w.WriteHeader(http.StatusOK) // TODO change to 204 No Content in new version } } @@ -236,7 +240,9 @@ func SnapshotOldGUIHandler(db *sqlx.DB, cfg config.Config) http.HandlerFunc { writePerlHTMLErr(w, r, err) return } - + if err := api.CreateChangeLogRaw(api.ApiChange, "Snapshot of CRConfig performed for "+cdn, *user, db.DB); err != nil { + log.Errorln("creating CRConfig snapshot changelog: " + err.Error()) + } http.Redirect(w, r, "/tools/flash_and_close/"+url.PathEscape("Successfully wrote the CRConfig.json!"), http.StatusFound) } } This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c opened a new pull request #2280: Fix TO CRconfig changelog
rob05c opened a new pull request #2280: Fix TO CRconfig changelog URL: https://github.com/apache/incubator-trafficcontrol/pull/2280 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rawlinp commented on issue #2279: Increase server hostname max length
rawlinp commented on issue #2279: Increase server hostname max length URL: https://github.com/apache/incubator-trafficcontrol/issues/2279#issuecomment-389668866 Yeah I think since the DB type is just `text` we should be able to bump up the artificial limit in TP. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jhg03a commented on issue #2279: Increase server hostname max length
jhg03a commented on issue #2279: Increase server hostname max length URL: https://github.com/apache/incubator-trafficcontrol/issues/2279#issuecomment-389668324 Yes. I am able to enter the data in the old TO interface, but that makes the server uneditable in the current TP. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 commented on issue #2279: Increase server hostname max length
mitchell852 commented on issue #2279: Increase server hostname max length URL: https://github.com/apache/incubator-trafficcontrol/issues/2279#issuecomment-389652233 Are you referring to the field in traffic portal? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jhg03a opened a new issue #2279: Increase server hostname max length
jhg03a opened a new issue #2279: Increase server hostname max length URL: https://github.com/apache/incubator-trafficcontrol/issues/2279 As a CDN SRE I want to be able to supply an arbitrarily long hostname. Today I actually only need 5 more characters, but this seems like a hidden limit that will just cause more problems later. Today I don't have this issue with the domainname field, but it would seem to be the same issue. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c closed issue #2243: Routes.go file should include type capture groups to enforce route parameter types
rob05c closed issue #2243: Routes.go file should include type capture groups to enforce route parameter types URL: https://github.com/apache/incubator-trafficcontrol/issues/2243 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] DylanVolz commented on issue #2277: Allow the capture group for a route parameter to be defined
DylanVolz commented on issue #2277: Allow the capture group for a route parameter to be defined URL: https://github.com/apache/incubator-trafficcontrol/pull/2277#issuecomment-389624176 After analysis of the legacy routes we will be porting from perl to go this feature is unnecessary. By following our own api guidelines in the future we shouldn't require this feature either. Closing this PR and issue #2243 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] DylanVolz closed pull request #2277: Allow the capture group for a route parameter to be defined
DylanVolz closed pull request #2277: Allow the capture group for a route parameter to be defined URL: https://github.com/apache/incubator-trafficcontrol/pull/2277 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/traffic_ops/traffic_ops_golang/routes.go b/traffic_ops/traffic_ops_golang/routes.go index 5763f703d..56ab8827b 100644 --- a/traffic_ops/traffic_ops_golang/routes.go +++ b/traffic_ops/traffic_ops_golang/routes.go @@ -79,17 +79,17 @@ func Routes(d ServerData) ([]Route, []RawRoute, http.Handler, error) { //ASN: CRUD {1.2, http.MethodGet, `asns/?(\.json)?$`, api.ReadHandler(asn.GetRefTypeV12(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, {1.1, http.MethodGet, `asns/?(\.json)?$`, asn.V11ReadAll(d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, - {1.1, http.MethodGet, `asns/{id}$`, api.ReadHandler(asn.GetRefTypeV11(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, - {1.1, http.MethodPut, `asns/{id}$`, api.UpdateHandler(asn.GetRefTypeV11(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, + {1.1, http.MethodGet, `asns/{id:[0-9]+}$`, api.ReadHandler(asn.GetRefTypeV11(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, + {1.1, http.MethodPut, `asns/{id:[0-9]+}$`, api.UpdateHandler(asn.GetRefTypeV11(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, {1.1, http.MethodPost, `asns/?$`, api.CreateHandler(asn.GetRefTypeV11(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, - {1.1, http.MethodDelete, `asns/{id}$`, api.DeleteHandler(asn.GetRefTypeV11(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, + {1.1, http.MethodDelete, `asns/{id:[0-9]+}$`, api.DeleteHandler(asn.GetRefTypeV11(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, //CacheGroup: CRUD {1.1, http.MethodGet, `cachegroups/?(\.json)?$`, api.ReadHandler(cachegroup.GetRefType(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, - {1.1, http.MethodGet, `cachegroups/{id}$`, api.ReadHandler(cachegroup.GetRefType(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, - {1.1, http.MethodPut, `cachegroups/{id}$`, api.UpdateHandler(cachegroup.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, + {1.1, http.MethodGet, `cachegroups/{id:[0-9]+}$`, api.ReadHandler(cachegroup.GetRefType(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, + {1.1, http.MethodPut, `cachegroups/{id:[0-9]+}$`, api.UpdateHandler(cachegroup.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, {1.1, http.MethodPost, `cachegroups/?$`, api.CreateHandler(cachegroup.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, - {1.1, http.MethodDelete, `cachegroups/{id}$`, api.DeleteHandler(cachegroup.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, + {1.1, http.MethodDelete, `cachegroups/{id:[0-9]+}$`, api.DeleteHandler(cachegroup.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, //CDN {1.1, http.MethodGet, `cdns/capacity$`, handlerToFunc(proxyHandler), 0, NoAuth, []Middleware{}}, @@ -100,57 +100,57 @@ func Routes(d ServerData) ([]Route, []RawRoute, http.Handler, error) { //CDN: CRUD {1.1, http.MethodGet, `cdns/?(\.json)?$`, api.ReadHandler(cdn.GetRefType(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, - {1.1, http.MethodGet, `cdns/{id}$`, api.ReadHandler(cdn.GetRefType(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, - {1.1, http.MethodPut, `cdns/{id}$`, api.UpdateHandler(cdn.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, + {1.1, http.MethodGet, `cdns/{id:[0-9]+}$`, api.ReadHandler(cdn.GetRefType(), d.DB), auth.PrivLevelReadOnly, Authenticated, nil}, + {1.1, http.MethodPut, `cdns/{id:[0-9]+}$`, api.UpdateHandler(cdn.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, {1.1, http.MethodPost, `cdns/?$`, api.CreateHandler(cdn.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, - {1.1, http.MethodDelete, `cdns/{id}$`, api.DeleteHandler(cdn.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, + {1.1, http.MethodDelete, `cdns/{id:[0-9]+}$`, api.DeleteHandler(cdn.GetRefType(), d.DB), auth.PrivLevelOperations, Authenticated, nil}, //CDN: Monitoring: Traffic Monitor {1.1, http.MethodGet, `cdns/{name}/configs/monitoring(\.json)?$`,
[GitHub] DylanVolz commented on issue #2243: Routes.go file should include type capture groups to enforce route parameter types
DylanVolz commented on issue #2243: Routes.go file should include type capture groups to enforce route parameter types URL: https://github.com/apache/incubator-trafficcontrol/issues/2243#issuecomment-389624250 After analysis of the legacy routes we will be porting from perl to go this feature is unnecessary. By following our own api guidelines in the future we shouldn't require this feature either. closing. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2247: Add an Origin API
asfgit commented on issue #2247: Add an Origin API URL: https://github.com/apache/incubator-trafficcontrol/pull/2247#issuecomment-389582209 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1585/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] knutsel commented on issue #2246: Add HitCount and FreshFor
knutsel commented on issue #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#issuecomment-389554197 @rob05c - like this? Sorry that the diff in rfc/rules.go is a bit bigger than needed, hard to get it all in the right place. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2246: Add HitCount and FreshFor
asfgit commented on issue #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#issuecomment-389553758 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1584/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2246: Add HitCount and FreshFor
asfgit commented on issue #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#issuecomment-389552212 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1583/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2246: Add HitCount and FreshFor
asfgit commented on issue #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#issuecomment-389530137 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1582/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] knutsel commented on a change in pull request #2246: Add HitCount and FreshFor
knutsel commented on a change in pull request #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#discussion_r188635843 ## File path: grove/plugin/http_cacheinspector.go ## @@ -169,16 +170,17 @@ func cacheinspect(icfg interface{}, d OnRequestData) bool { w.Write([]byte(fmt.Sprintf("showing only first %d and last %d:\n\n", head, tail))) } - w.Write([]byte(fmt.Sprintf(" #\t\tCode\tSize\tAge\t\t\tKey\n"))) + w.Write([]byte(fmt.Sprintf("#Code Size Age FreshForHitCount Key\n"))) for i, key := range keys { - if (doSearch && !strings.Contains(key, searchArr[0])) || !doSearch && (i > tail && i < len(keys)-head) { + if (doSearch && !strings.Contains(key, searchArr[0])) || !doSearch && (i >= tail && i < len(keys)-head) { continue } cacheObject, _ := d.Stats.CachePeek(key, cName) age := time.Now().Sub(cacheObject.ReqRespTime) - w.Write([]byte(fmt.Sprintf(" %05d\t%d\t%s\t%-20v\thttp://%s%s?key=%s=%s\;>%s\n", - i, cacheObject.Code, bytefmt.ByteSize(cacheObject.Size), age, req.Host, CacheStatsEndpoint, url.QueryEscape(key), cName, key))) + freshFor := web.FreshFor(cacheObject.RespHeaders, cacheObject.RespCacheControl, cacheObject.ReqRespTime, cacheObject.RespRespTime) + w.Write([]byte(fmt.Sprintf(" %8d%8d%10s%22v%22v%12d http://%s%s?key=%s=%s\;>%s\n", Review comment: It compiles, and works, so I'm going with it ;-) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2277: Allow the capture group for a route parameter to be defined
asfgit commented on issue #2277: Allow the capture group for a route parameter to be defined URL: https://github.com/apache/incubator-trafficcontrol/pull/2277#issuecomment-389365591 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1581/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2278: Add TO Go caches/stats endpoint
asfgit commented on issue #2278: Add TO Go caches/stats endpoint URL: https://github.com/apache/incubator-trafficcontrol/pull/2278#issuecomment-389355518 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1580/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2108: create roles crud endpoints in golang
asfgit commented on issue #2108: create roles crud endpoints in golang URL: https://github.com/apache/incubator-trafficcontrol/pull/2108#issuecomment-389347426 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1579/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on issue #2232: Rewrite Traffic Ops CRConfig Table Endpoints in Go
rob05c commented on issue #2232: Rewrite Traffic Ops CRConfig Table Endpoints in Go URL: https://github.com/apache/incubator-trafficcontrol/issues/2232#issuecomment-387444293 Ok, we have PRs for all major table routes, but many of them are only "CRUD", and missing additional endpoints. So I'm listing all Perl API routes here. * if you've verified a route does not use one of the CRConfig tables (above), follow it with a bullet noting that. Be sure to verify any tables the route does use do not link a CRConfig table, in those tables' `traffic_ops/app/lib/Schema/Result` * if there is an unmerged PR, follow with a bullet linking the PR * if the route is in Go, in master, check it off - [x] get /api/1.1/asns - [x] get /api/1.2/asns - [x] get /api/1.1/asns/:id - [x] post /api/1.1/asns - [x] put /api/1.1/asns/:id - [x] delete /api/1.1/asns/:id - [ ] get /api/1.1/caches/stats - https://github.com/apache/incubator-trafficcontrol/pull/2278 - [x] get /api/1.1/cachegroups - [ ] get /api/1.1/cachegroups/trimmed - [x] get /api/1.1/cachegroups/:id - [x] post /api/1.1/cachegroups - [x] put /api/1.1/cachegroups/:id - [x] delete /api/1.1/cachegroups/:id - [ ] post /api/1.1/cachegroups/:id/deliveryservices - [ ] post /api/1.1/cachegroups/:id/queue_update - [x] get /api/1.1/cdns - [x] get /api/1.1/cdns/:id - [ ] get /api/1.1/cdns/name/:name - [x] post /api/1.1/cdns - [x] put /api/1.1/cdns/:id - [x] delete /api/1.1/cdns/:id - [ ] delete /api/1.1/cdns/name/:name - [ ] post /api/1.1/cdns/:id/queue_update - [ ] get /api/1.1/cdns/health - [ ] get /api/1.1/cdns/:name/health - [ ] get /api/1.1/cdns/capacity - [ ] get /api/1.1/cdns/routing - [x] get /api/1.1/cdns/:name/snapshot - [x] get /api/1.1/cdns/:name/snapshot/new - [x] put /api/1.1/cdns/:id/snapshot - [x] put /api/1.1/snapshot/:cdn_name - [ ] get /api/1.1/cdns/metric_types/:metric_type/start_date/:start_date/end_date/:end_date - [ ] get /api/1.1/cdns/name/:name/dnsseckeys - [ ] post /api/1.1/cdns/dnsseckeys/generate - [ ] get /api/1.1/cdns/name/:name/dnsseckeys/delete - [ ] get /internal/api/1.1/cdns/dnsseckeys/refresh - [ ] get /api/1.1/cdns/name/:name/sslkeys - [ ] get /api/1.1/cdns/configs - [ ] get /api/1.1/cdns/:name/configs/routing - [x] get /api/1.1/cdns/:name/configs/monitoring - [ ] get /api/1.1/cdns/domains - [ ] get /api/1.1/logs - [ ] get /api/1.1/logs/:days/days - [ ] get /api/1.1/logs/newcount - [ ] get /api/1.1/servers/:id/configfiles/ats - [ ] get /api/1.1/profiles/:id/configfiles/ats/:filename - [ ] get /api/1.1/servers/:id/configfiles/ats/:filename - [ ] get /api/1.1/cdns/:id/configfiles/ats/:filename - [ ] get /api/1.1/dbdump - [ ] get /api/1.1/deliveryservices - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [ ] get /api/1.1/deliveryservices/:id - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [ ] post /api/1.1/deliveryservices - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [ ] put /api/1.1/deliveryservices/:id - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [ ] put /api/1.1/deliveryservices/:id/safe - [ ] delete /api/1.1/deliveryservices/:id - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [x] get /api/1.1/servers/:id/deliveryservices - [ ] post /api/1.1/deliveryservices/:xml_id/servers - [ ] delete /api/1.1/deliveryservice_server/:dsId/:serverId - [ ] get /api/1.1/deliveryservices/:id/health - [ ] get /api/1.1/deliveryservices/:id/capacity - [ ] get /api/1.1/deliveryservices/:id/routing - [ ] get /api/1.1/deliveryservices/:id/state - [ ] post /api/1.1/deliveryservices/request - [ ] get /internal/api/1.1/steering - [ ] get /internal/api/1.1/steering/:xml_id - [ ] put /internal/api/1.1/steering/:xml_id - [ ] get /api/1.1/steering/:id/targets - [ ] get /api/1.1/steering/:id/targets/:target_id - [ ] post /api/1.1/steering/:id/targets - [ ] put /api/1.1/steering/:id/targets/:target_id - [ ] delete /api/1.1/steering/:id/targets/:target_id - [x] get /api/1.1/deliveryservices/xmlId/:xmlid/sslkeys - [ ] post /api/1.1/deliveryservices/sslkeys/generate - [x] post /api/1.1/deliveryservices/sslkeys/add - [ ] get /api/1.1/deliveryservices/xmlId/:xmlid/sslkeys/delete - [ ] post /api/1.1/deliveryservices/xmlId/:xmlId/urlkeys/generate - [ ] post /api/1.1/deliveryservices/xmlId/:xmlId/urlkeys/copyFromXmlId/:copyFromXmlId - [ ] get /api/1.1/deliveryservices/xmlId/:xmlId/urlkeys - [ ] get /api/1.1/deliveryservices/:id/urlkeys - [ ] get /api/1.1/deliveryservices_regexes - PR https://github.com/apache/incubator-trafficcontrol/pull/2229 - [ ] get /api/1.1/deliveryservices/:dsId/regexes - PR https://github.com/apache/incubator-trafficcontrol/pull/2229 - [ ] get
[GitHub] rob05c commented on issue #2277: Allow the capture group for a route parameter to be defined
rob05c commented on issue #2277: Allow the capture group for a route parameter to be defined URL: https://github.com/apache/incubator-trafficcontrol/pull/2277#issuecomment-389339315 I'm still not convinced this is worth the readability cost. Regex is hard to read. This makes the routes harder to read, and the router more complex, for what gain? It's pretty common for HTTP dispatch to be in-order, I don't think it's unreasonable for developers to be required to understand that routes match in order. Or for the route to check the param type, with a helper func, it's a single error check, like https://github.com/apache/incubator-trafficcontrol/pull/2256/files#diff-c99ae9948456bba1d5438e02608d7bc7R33 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on issue #2277: Allow the capture group for a route parameter to be defined
rob05c commented on issue #2277: Allow the capture group for a route parameter to be defined URL: https://github.com/apache/incubator-trafficcontrol/pull/2277#issuecomment-389339315 I'm still not convinced this is worth the readability cost. Regex is hard to read. This makes the routes harder to read, and the router more complex, for what gain? It's pretty common for HTTP dispatch to be in-order, I don't think it's unreasonable for developers to be required to understand that routes match in order. Or for the route to check the param type, with a helper, it's a single error check, like https://github.com/apache/incubator-trafficcontrol/pull/2256/files#diff-c99ae9948456bba1d5438e02608d7bc7R33 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] DylanVolz commented on issue #2273: TO golang API framework tenancy enhancements
DylanVolz commented on issue #2273: TO golang API framework tenancy enhancements URL: https://github.com/apache/incubator-trafficcontrol/issues/2273#issuecomment-389337746 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c opened a new pull request #2278: Add TO Go caches/stats endpoint
rob05c opened a new pull request #2278: Add TO Go caches/stats endpoint URL: https://github.com/apache/incubator-trafficcontrol/pull/2278 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] DylanVolz opened a new pull request #2277: Allow the capture group for a route parameter to be defined
DylanVolz opened a new pull request #2277: Allow the capture group for a route parameter to be defined URL: https://github.com/apache/incubator-trafficcontrol/pull/2277 this is done in the form {parameterName:capturePattern} instead of {parameterName} in the route definition in routes.go. The capture pattern is defined using a regex pattern the same as the rest of the route definition. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2236: update portal info bubbles
asfgit commented on issue #2236: update portal info bubbles URL: https://github.com/apache/incubator-trafficcontrol/pull/2236#issuecomment-389334635 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1578/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2236: update portal info bubbles
asfgit commented on issue #2236: update portal info bubbles URL: https://github.com/apache/incubator-trafficcontrol/pull/2236#issuecomment-389322775 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1577/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2276: Migration to add anonymousBlockingEnabled field to deliveryservice_request json blob data
asfgit commented on issue #2276: Migration to add anonymousBlockingEnabled field to deliveryservice_request json blob data URL: https://github.com/apache/incubator-trafficcontrol/pull/2276#issuecomment-389318776 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2276: Migration to add anonymousBlockingEnabled field to deliveryservice_request json blob data
asfgit commented on issue #2276: Migration to add anonymousBlockingEnabled field to deliveryservice_request json blob data URL: https://github.com/apache/incubator-trafficcontrol/pull/2276#issuecomment-389318472 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rivasj opened a new pull request #2276: Migration to add anonymousBlockingEnabled field to deliveryservice_request json blob data
rivasj opened a new pull request #2276: Migration to add anonymousBlockingEnabled field to deliveryservice_request json blob data URL: https://github.com/apache/incubator-trafficcontrol/pull/2276 Add anonymousBlockingEnabled field to deliveryservice_request json blob data in db (deliveryservice_request.deliveryservice) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2247: Add an Origin API
asfgit commented on issue #2247: Add an Origin API URL: https://github.com/apache/incubator-trafficcontrol/pull/2247#issuecomment-389307651 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1576/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on issue #2274: Profiles endpoint unordered
rob05c commented on issue #2274: Profiles endpoint unordered URL: https://github.com/apache/incubator-trafficcontrol/issues/2274#issuecomment-389306244 Ordering in the database can be expensive, and the API has never guaranteed order. See https://github.com/apache/incubator-trafficcontrol/issues/2259 I'm -1 on sorting in the database; better to put the workload on clients/browsers, than the server. If you need to order, most languages can easily sort. On the command line, `curl -Lvsk --cookie $mc 'https://localhost/api/1.2/profiles' | jq '.response | sort_by(.name)'` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dneuman64 opened a new issue #2275: Download ISO should actually download the ISO
dneuman64 opened a new issue #2275: Download ISO should actually download the ISO URL: https://github.com/apache/incubator-trafficcontrol/issues/2275 Currently, when you generate an ISO you are given a link to download the ISO. If you are running a traffic ops in a HA fashion, it is no guarantee that the link will send you to the same Traffic Ops meaning that there is a possibility that you will not be able to download the ISO. Instead of giving the user a link to download the ISO, traffic ops should just let the user download the ISO. This would resolve the issue where you could get sent to a different Traffic Ops to download the ISO. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jhg03a opened a new issue #2274: Profiles endpoint unordered
jhg03a opened a new issue #2274: Profiles endpoint unordered URL: https://github.com/apache/incubator-trafficcontrol/issues/2274 With the Golang rewrite, the profiles endpoint is no longer ordered by name. This makes the dialogs such as in delivery services more difficult to navigate. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jhg03a closed issue #2263: Improve TO Client Library Documentation
jhg03a closed issue #2263: Improve TO Client Library Documentation URL: https://github.com/apache/incubator-trafficcontrol/issues/2263 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 closed issue #886: [TC-287] Add support for per-Delivery Service routing/name
mitchell852 closed issue #886: [TC-287] Add support for per-Delivery Service routing/name URL: https://github.com/apache/incubator-trafficcontrol/issues/886 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rawlinp commented on issue #886: [TC-287] Add support for per-Delivery Service routing/name
rawlinp commented on issue #886: [TC-287] Add support for per-Delivery Service routing/name URL: https://github.com/apache/incubator-trafficcontrol/issues/886#issuecomment-389292459 This can be closed - implemented in #865 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188122792 ## File path: grove/plugin/range_req_handler.go ## @@ -0,0 +1,225 @@ +package plugin + +/* + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/apache/incubator-trafficcontrol/grove/web" + "github.com/apache/incubator-trafficcontrol/lib/go-log" +) + +type byteRange struct { + Start int64 + End int64 +} + +type rangeRequestConfig struct { + Mode string `json::"mode"` +} + +func init() { + AddPlugin(1, Funcs{ + load:rangeReqHandleLoad, + onRequest: rangeReqHandlerOnRequest, + beforeCacheLookUp: rangeReqHandleBeforeCacheLookup, + beforeParentRequest: rangeReqHandleBeforeParent, + beforeRespond: rangeReqHandleBeforeRespond, + }) +} + +// rangeReqHandleLoad loads the configuration +func rangeReqHandleLoad(b json.RawMessage) interface{} { + cfg := rangeRequestConfig{} + log.Errorf("rangeReqHandleLoad loading: %s", b) + + err := json.Unmarshal(b, ) + if err != nil { + log.Errorln("range_rew_handler loading config, unmarshalling JSON: " + err.Error()) + return nil + } + if !(cfg.Mode == "get_full_serve_range" || cfg.Mode == "patch") { + log.Errorf("Unknown mode for range_req_handler plugin: %s", cfg.Mode) + } + log.Debugf("range_rew_handler: load success: %+v\n", cfg) + return +} + +// rangeReqHandlerOnRequest determines if there is a Range header, and puts the ranges in *d.Context as a []byteRanges +func rangeReqHandlerOnRequest(icfg interface{}, d OnRequestData) bool { + rHeader := d.R.Header.Get("Range") + if rHeader == "" { + log.Debugf("No Range header found") + return false + } + log.Debugf("Range string is: %s", rHeader) + // put the ranges [] in the context so we can use it later + byteRanges := parseRangeHeader(rHeader) + *d.Context = byteRanges + return false +} + +func rangeReqHandleBeforeCacheLookup(icfg interface{}, d BeforeCacheLookUpData, overRideFunc func(string)) { + cfg, ok := icfg.(*rangeRequestConfig) + if !ok { + log.Errorf("range_req_handler config '%v' type '%T' expected *rangeRequestConfig\n", icfg, icfg) + return + } + if cfg.Mode == "store_ranges" { + sep := "?" + if strings.Contains(d.DefaultCacheKey, "?") { + sep = "&" + } + newKey := d.DefaultCacheKey + sep + "grove_range_req_handler_plugin_data=" + d.Req.Header.Get("Range") + overRideFunc(newKey) + log.Debugf("range_req_handler: store_ranges default key:%s, new key:%s", d.DefaultCacheKey, newKey) + } +} + +// rangeReqHandleBeforeParent changes the parent request if needed (mode == get_full_serve_range) +func rangeReqHandleBeforeParent(icfg interface{}, d BeforeParentRequestData) { + log.Debugf("rangeReqHandleBeforeParent calling.") + rHeader := d.Req.Header.Get("Range") + if rHeader == "" { + log.Debugf("No Range header found") + return + } + log.Debugf("Range string is: %s", rHeader) + cfg, ok := icfg.(*rangeRequestConfig) + if !ok { + log.Errorf("range_req_handler config '%v' type '%T' expected *rangeRequestConfig\n", icfg, icfg) + return + } + if cfg.Mode == "get_full_serve_range" { + // get_full_serve_range means get the whole thing from parent/org, but serve the requested range. Just remove the Range header from the upstream request + d.Req.Header.Del("Range") + } + return +} + +// rangeReqHandleBeforeRespond builds the 206 response +// Assume all the needed ranges have been put in cache before, which is the truth for "get_full_serve_range" mode which gets the whole object into cache. +func rangeReqHandleBeforeRespond(icfg interface{}, d BeforeRespondData) { + log.Debugf("rangeReqHandleBeforeRespond calling\n") + ictx
[GitHub] rawlinp opened a new issue #2273: TO golang API framework tenancy enhancements
rawlinp opened a new issue #2273: TO golang API framework tenancy enhancements URL: https://github.com/apache/incubator-trafficcontrol/issues/2273 While working on the Origin API, I found a few things related to tenancy in the TO golang API framework that should be enhanced: 1. The shared handler functions (e.g. UpdateHandler, CreateHandler, etc.) should check if tenancy is enabled before calling `IsTenantAuthorized` on a `Tenantable` type. This prevents each `Delete`, `Update`, etc. function from having to check if tenancy is actually enabled before checking further. Generally if tenancy is disabled there should be no checking of tenancy. This could be cached on startup so that it doesn't have to be checked on every single request. 2. If the `isTenantAuthorized` function had an added `operation enum` parameter added (i.e. when calling it from the `CreateHandler`, it would pass `Operation.Create`, from the `DeleteHandler` it would pass `Operation.Delete`, etc), you could add specialized tenancy logic to your `isTenantAuthorized` implementation for the type of operation being performed. For instance, `Delete()` doesn't take a request body, so you only need to check if the user has access to the existing tenant. However, for `Update()`, you need to check the tenancy on both the current and requested tenants. It would be better if all the tenancy logic was able to stay in `isTenantAuthorized` rather than specialized tenancy checks in each method. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2010: Go login
asfgit commented on issue #2010: Go login URL: https://github.com/apache/incubator-trafficcontrol/pull/2010#issuecomment-389214762 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1575/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2236: update portal info bubbles
asfgit commented on issue #2236: update portal info bubbles URL: https://github.com/apache/incubator-trafficcontrol/pull/2236#issuecomment-389209826 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1574/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] elsloo opened a new issue #2272: Only create/update/modify DNSSEC keys for DS types that are routed on the CDN
elsloo opened a new issue #2272: Only create/update/modify DNSSEC keys for DS types that are routed on the CDN URL: https://github.com/apache/incubator-trafficcontrol/issues/2272 Create DNSSEC keys only for DS types that are routed on the CDN, such as any HTTP or DNS type. ANY_MAP, or other types that are not routed on the CDN should not have DNSSEC keys created for them. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2246: Add HitCount and FreshFor
rob05c commented on a change in pull request #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#discussion_r188317853 ## File path: grove/web/util.go ## @@ -198,3 +200,145 @@ func ParseHTTPDate(d string) (time.Time, bool) { // RemapTextKey is the plugin shared data key inserted by grovetccfg for the Remap Line of the Delivery Service in Traffic Control, Traffic Ops. const RemapTextKey = "remap_text" + +const Day = time.Hour * time.Duration(24) + +// GetHTTPDeltaSeconds is a helper function which gets an HTTP Delta Seconds from the given map (which is typically a `http.Header` or `CacheControl`. Returns false if the given key doesn't exist in the map, or if the value isn't a valid Delta Seconds per RFC2616§3.3.2. +func GetHTTPDeltaSeconds(m map[string][]string, key string) (time.Duration, bool) { Review comment: How about `rfc`? Can you add a Godoc package-level comment too, along the lines of ``` // Package rfc contains functions implementing RFC 7234, 2616, and other RFCs. // When changing functions, be sure they still conform to the corresponding RFC. // When adding symbols, document the RFC and section they correspond to. package rfc ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2246: Add HitCount and FreshFor
rob05c commented on a change in pull request #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#discussion_r188317853 ## File path: grove/web/util.go ## @@ -198,3 +200,145 @@ func ParseHTTPDate(d string) (time.Time, bool) { // RemapTextKey is the plugin shared data key inserted by grovetccfg for the Remap Line of the Delivery Service in Traffic Control, Traffic Ops. const RemapTextKey = "remap_text" + +const Day = time.Hour * time.Duration(24) + +// GetHTTPDeltaSeconds is a helper function which gets an HTTP Delta Seconds from the given map (which is typically a `http.Header` or `CacheControl`. Returns false if the given key doesn't exist in the map, or if the value isn't a valid Delta Seconds per RFC2616§3.3.2. +func GetHTTPDeltaSeconds(m map[string][]string, key string) (time.Duration, bool) { Review comment: How about `rfc`? Can you add a Godoc package-level comment too, along the lines of ``` // Package rfc contains functions implementing RFC 7234, 2616, and other RFCs // When changing functions, be sure they still conform to the corresponding RFC. // When adding symbols, document the RFC and section they correspond to. package rfc ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 commented on a change in pull request #2029: [Issue 1907] TO API for backup edge cachegroup
mitchell852 commented on a change in pull request #2029: [Issue 1907] TO API for backup edge cachegroup URL: https://github.com/apache/incubator-trafficcontrol/pull/2029#discussion_r188317346 ## File path: traffic_ops/app/db/migrations/2018032100_cache_group_fallback.sql ## @@ -0,0 +1,38 @@ +/* + Review comment: this migration won't work. the date is too old. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 commented on a change in pull request #2029: [Issue 1907] TO API for backup edge cachegroup
mitchell852 commented on a change in pull request #2029: [Issue 1907] TO API for backup edge cachegroup URL: https://github.com/apache/incubator-trafficcontrol/pull/2029#discussion_r188315094 ## File path: traffic_ops/app/lib/UI/Topology.pm ## @@ -250,6 +250,15 @@ sub gen_crconfig_json { if ( $row->type->name =~ m/^EDGE/ ) { $data_obj->{'edgeLocations'}->{ $row->cachegroup->name }->{'latitude'} = $row->cachegroup->latitude + 0; $data_obj->{'edgeLocations'}->{ $row->cachegroup->name }->{'longitude'} = $row->cachegroup->longitude + 0; +$data_obj->{'edgeLocations'}->{ $row->cachegroup->name }->{'backupLocations'}->{'fallbackToClosest'} = $row->cachegroup->fallback_to_closest ? "true" : "false"; + +my $rs_backups = $self->db->resultset('CachegroupFallback')->search({ primary_cg => $row->cachegroup->id}, {order_by => 'set_order'}); Review comment: gen_crconfig_json was rewritten in Golang so this logic will need to be added there as well otherwise it will be lost. You might want to reach out to @rob05c This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 commented on issue #2029: [Issue 1907] TO API for backup edge cachegroup
mitchell852 commented on issue #2029: [Issue 1907] TO API for backup edge cachegroup URL: https://github.com/apache/incubator-trafficcontrol/pull/2029#issuecomment-389192781 this seems ok from an api perspective. need someone to review/test the TR part. @rawlinp @rivasj @elsloo @ajschmidt any takers? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188314219 ## File path: grove/plugin/range_req_handler.go ## @@ -0,0 +1,225 @@ +package plugin + +/* + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/apache/incubator-trafficcontrol/grove/web" + "github.com/apache/incubator-trafficcontrol/lib/go-log" +) + +type byteRange struct { + Start int64 + End int64 +} + +type rangeRequestConfig struct { + Mode string `json::"mode"` +} + +func init() { + AddPlugin(1, Funcs{ + load:rangeReqHandleLoad, + onRequest: rangeReqHandlerOnRequest, + beforeCacheLookUp: rangeReqHandleBeforeCacheLookup, + beforeParentRequest: rangeReqHandleBeforeParent, + beforeRespond: rangeReqHandleBeforeRespond, + }) +} + +// rangeReqHandleLoad loads the configuration +func rangeReqHandleLoad(b json.RawMessage) interface{} { + cfg := rangeRequestConfig{} + log.Errorf("rangeReqHandleLoad loading: %s", b) + + err := json.Unmarshal(b, ) + if err != nil { + log.Errorln("range_rew_handler loading config, unmarshalling JSON: " + err.Error()) + return nil + } + if !(cfg.Mode == "get_full_serve_range" || cfg.Mode == "patch") { + log.Errorf("Unknown mode for range_req_handler plugin: %s", cfg.Mode) + } + log.Debugf("range_rew_handler: load success: %+v\n", cfg) + return +} + +// rangeReqHandlerOnRequest determines if there is a Range header, and puts the ranges in *d.Context as a []byteRanges +func rangeReqHandlerOnRequest(icfg interface{}, d OnRequestData) bool { + rHeader := d.R.Header.Get("Range") + if rHeader == "" { + log.Debugf("No Range header found") + return false + } + log.Debugf("Range string is: %s", rHeader) + // put the ranges [] in the context so we can use it later + byteRanges := parseRangeHeader(rHeader) + *d.Context = byteRanges + return false +} + +func rangeReqHandleBeforeCacheLookup(icfg interface{}, d BeforeCacheLookUpData, overRideFunc func(string)) { + cfg, ok := icfg.(*rangeRequestConfig) + if !ok { + log.Errorf("range_req_handler config '%v' type '%T' expected *rangeRequestConfig\n", icfg, icfg) + return + } + if cfg.Mode == "store_ranges" { + sep := "?" + if strings.Contains(d.DefaultCacheKey, "?") { + sep = "&" + } + newKey := d.DefaultCacheKey + sep + "grove_range_req_handler_plugin_data=" + d.Req.Header.Get("Range") + overRideFunc(newKey) + log.Debugf("range_req_handler: store_ranges default key:%s, new key:%s", d.DefaultCacheKey, newKey) + } +} + +// rangeReqHandleBeforeParent changes the parent request if needed (mode == get_full_serve_range) +func rangeReqHandleBeforeParent(icfg interface{}, d BeforeParentRequestData) { + log.Debugf("rangeReqHandleBeforeParent calling.") + rHeader := d.Req.Header.Get("Range") + if rHeader == "" { + log.Debugf("No Range header found") + return + } + log.Debugf("Range string is: %s", rHeader) + cfg, ok := icfg.(*rangeRequestConfig) + if !ok { + log.Errorf("range_req_handler config '%v' type '%T' expected *rangeRequestConfig\n", icfg, icfg) + return + } + if cfg.Mode == "get_full_serve_range" { + // get_full_serve_range means get the whole thing from parent/org, but serve the requested range. Just remove the Range header from the upstream request + d.Req.Header.Del("Range") + } + return +} + +// rangeReqHandleBeforeRespond builds the 206 response +// Assume all the needed ranges have been put in cache before, which is the truth for "get_full_serve_range" mode which gets the whole object into cache. +func rangeReqHandleBeforeRespond(icfg interface{}, d BeforeRespondData) { + log.Debugf("rangeReqHandleBeforeRespond calling\n") + ictx
[GitHub] asfgit commented on issue #2271: if user.role == admin, then allow the option to immediately fulfill DSR (d…
asfgit commented on issue #2271: if user.role == admin, then allow the option to immediately fulfill DSR (d… URL: https://github.com/apache/incubator-trafficcontrol/pull/2271#issuecomment-389192028 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1573/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 commented on issue #2029: [Issue 1907] TO API for backup edge cachegroup
mitchell852 commented on issue #2029: [Issue 1907] TO API for backup edge cachegroup URL: https://github.com/apache/incubator-trafficcontrol/pull/2029#issuecomment-389185605 yes to last comment. that way POST is reserved for creating and PUT is reserved for updating and they don’t overlap. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 commented on issue #2174: TO/TP should allow for a configuration such that admins do not have to submit DSRs
mitchell852 commented on issue #2174: TO/TP should allow for a configuration such that admins do not have to submit DSRs URL: https://github.com/apache/incubator-trafficcontrol/issues/2174#issuecomment-389181872 Implemented as such: If DSR is enabled and user.role=admin, then DSR is created/closed in the background. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mitchell852 opened a new pull request #2271: if user.role == admin, then allow the option to immediately fulfill DSR (d…
mitchell852 opened a new pull request #2271: if user.role == admin, then allow the option to immediately fulfill DSR (d… URL: https://github.com/apache/incubator-trafficcontrol/pull/2271 …elivery service requests) fixes #2174 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] knutsel commented on a change in pull request #2246: Add HitCount and FreshFor
knutsel commented on a change in pull request #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#discussion_r188269492 ## File path: grove/web/util.go ## @@ -198,3 +200,145 @@ func ParseHTTPDate(d string) (time.Time, bool) { // RemapTextKey is the plugin shared data key inserted by grovetccfg for the Remap Line of the Delivery Service in Traffic Control, Traffic Ops. const RemapTextKey = "remap_text" + +const Day = time.Hour * time.Duration(24) + +// GetHTTPDeltaSeconds is a helper function which gets an HTTP Delta Seconds from the given map (which is typically a `http.Header` or `CacheControl`. Returns false if the given key doesn't exist in the map, or if the value isn't a valid Delta Seconds per RFC2616§3.3.2. +func GetHTTPDeltaSeconds(m map[string][]string, key string) (time.Duration, bool) { Review comment: I needed it outside the package, and it was lowercase in rules.go. I read rules as in remap-rules, not rfc-rules... So I thought web would be a better package, because I thought they're more general than remap-rules... +1 on making it's own package, want me to give that a try? What would you like to call it? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2088: TO Go: implement tenants CRUD
asfgit commented on issue #2088: TO Go: implement tenants CRUD URL: https://github.com/apache/incubator-trafficcontrol/pull/2088#issuecomment-389036164 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1572/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2088: TO Go: implement tenants CRUD
asfgit commented on issue #2088: TO Go: implement tenants CRUD URL: https://github.com/apache/incubator-trafficcontrol/pull/2088#issuecomment-389032173 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1571/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2247: Add an Origin API
asfgit commented on issue #2247: Add an Origin API URL: https://github.com/apache/incubator-trafficcontrol/pull/2247#issuecomment-388998048 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1570/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2247: Add an Origin API
asfgit commented on issue #2247: Add an Origin API URL: https://github.com/apache/incubator-trafficcontrol/pull/2247#issuecomment-388997317 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1569/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2270: Add TO Go Servers Details endpoint
asfgit commented on issue #2270: Add TO Go Servers Details endpoint URL: https://github.com/apache/incubator-trafficcontrol/pull/2270#issuecomment-388989863 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1568/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188122792 ## File path: grove/plugin/range_req_handler.go ## @@ -0,0 +1,225 @@ +package plugin + +/* + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/apache/incubator-trafficcontrol/grove/web" + "github.com/apache/incubator-trafficcontrol/lib/go-log" +) + +type byteRange struct { + Start int64 + End int64 +} + +type rangeRequestConfig struct { + Mode string `json::"mode"` +} + +func init() { + AddPlugin(1, Funcs{ + load:rangeReqHandleLoad, + onRequest: rangeReqHandlerOnRequest, + beforeCacheLookUp: rangeReqHandleBeforeCacheLookup, + beforeParentRequest: rangeReqHandleBeforeParent, + beforeRespond: rangeReqHandleBeforeRespond, + }) +} + +// rangeReqHandleLoad loads the configuration +func rangeReqHandleLoad(b json.RawMessage) interface{} { + cfg := rangeRequestConfig{} + log.Errorf("rangeReqHandleLoad loading: %s", b) + + err := json.Unmarshal(b, ) + if err != nil { + log.Errorln("range_rew_handler loading config, unmarshalling JSON: " + err.Error()) + return nil + } + if !(cfg.Mode == "get_full_serve_range" || cfg.Mode == "patch") { + log.Errorf("Unknown mode for range_req_handler plugin: %s", cfg.Mode) + } + log.Debugf("range_rew_handler: load success: %+v\n", cfg) + return +} + +// rangeReqHandlerOnRequest determines if there is a Range header, and puts the ranges in *d.Context as a []byteRanges +func rangeReqHandlerOnRequest(icfg interface{}, d OnRequestData) bool { + rHeader := d.R.Header.Get("Range") + if rHeader == "" { + log.Debugf("No Range header found") + return false + } + log.Debugf("Range string is: %s", rHeader) + // put the ranges [] in the context so we can use it later + byteRanges := parseRangeHeader(rHeader) + *d.Context = byteRanges + return false +} + +func rangeReqHandleBeforeCacheLookup(icfg interface{}, d BeforeCacheLookUpData, overRideFunc func(string)) { + cfg, ok := icfg.(*rangeRequestConfig) + if !ok { + log.Errorf("range_req_handler config '%v' type '%T' expected *rangeRequestConfig\n", icfg, icfg) + return + } + if cfg.Mode == "store_ranges" { + sep := "?" + if strings.Contains(d.DefaultCacheKey, "?") { + sep = "&" + } + newKey := d.DefaultCacheKey + sep + "grove_range_req_handler_plugin_data=" + d.Req.Header.Get("Range") + overRideFunc(newKey) + log.Debugf("range_req_handler: store_ranges default key:%s, new key:%s", d.DefaultCacheKey, newKey) + } +} + +// rangeReqHandleBeforeParent changes the parent request if needed (mode == get_full_serve_range) +func rangeReqHandleBeforeParent(icfg interface{}, d BeforeParentRequestData) { + log.Debugf("rangeReqHandleBeforeParent calling.") + rHeader := d.Req.Header.Get("Range") + if rHeader == "" { + log.Debugf("No Range header found") + return + } + log.Debugf("Range string is: %s", rHeader) + cfg, ok := icfg.(*rangeRequestConfig) + if !ok { + log.Errorf("range_req_handler config '%v' type '%T' expected *rangeRequestConfig\n", icfg, icfg) + return + } + if cfg.Mode == "get_full_serve_range" { + // get_full_serve_range means get the whole thing from parent/org, but serve the requested range. Just remove the Range header from the upstream request + d.Req.Header.Del("Range") + } + return +} + +// rangeReqHandleBeforeRespond builds the 206 response +// Assume all the needed ranges have been put in cache before, which is the truth for "get_full_serve_range" mode which gets the whole object into cache. +func rangeReqHandleBeforeRespond(icfg interface{}, d BeforeRespondData) { + log.Debugf("rangeReqHandleBeforeRespond calling\n") + ictx
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188121922 ## File path: grove/plugin/range_req_handler.go ## @@ -0,0 +1,225 @@ +package plugin + +/* + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/apache/incubator-trafficcontrol/grove/web" + "github.com/apache/incubator-trafficcontrol/lib/go-log" +) + +type byteRange struct { + Start int64 + End int64 +} + +type rangeRequestConfig struct { + Mode string `json::"mode"` +} + +func init() { + AddPlugin(1, Funcs{ + load:rangeReqHandleLoad, + onRequest: rangeReqHandlerOnRequest, + beforeCacheLookUp: rangeReqHandleBeforeCacheLookup, + beforeParentRequest: rangeReqHandleBeforeParent, + beforeRespond: rangeReqHandleBeforeRespond, + }) +} + +// rangeReqHandleLoad loads the configuration +func rangeReqHandleLoad(b json.RawMessage) interface{} { + cfg := rangeRequestConfig{} + log.Errorf("rangeReqHandleLoad loading: %s", b) + + err := json.Unmarshal(b, ) + if err != nil { + log.Errorln("range_rew_handler loading config, unmarshalling JSON: " + err.Error()) + return nil + } + if !(cfg.Mode == "get_full_serve_range" || cfg.Mode == "patch") { + log.Errorf("Unknown mode for range_req_handler plugin: %s", cfg.Mode) Review comment: Newline This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188121895 ## File path: grove/plugin/range_req_handler.go ## @@ -0,0 +1,225 @@ +package plugin + +/* + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/apache/incubator-trafficcontrol/grove/web" + "github.com/apache/incubator-trafficcontrol/lib/go-log" +) + +type byteRange struct { + Start int64 + End int64 +} + +type rangeRequestConfig struct { + Mode string `json::"mode"` +} + +func init() { + AddPlugin(1, Funcs{ + load:rangeReqHandleLoad, + onRequest: rangeReqHandlerOnRequest, + beforeCacheLookUp: rangeReqHandleBeforeCacheLookup, + beforeParentRequest: rangeReqHandleBeforeParent, + beforeRespond: rangeReqHandleBeforeRespond, + }) +} + +// rangeReqHandleLoad loads the configuration +func rangeReqHandleLoad(b json.RawMessage) interface{} { + cfg := rangeRequestConfig{} + log.Errorf("rangeReqHandleLoad loading: %s", b) + + err := json.Unmarshal(b, ) + if err != nil { + log.Errorln("range_rew_handler loading config, unmarshalling JSON: " + err.Error()) + return nil + } + if !(cfg.Mode == "get_full_serve_range" || cfg.Mode == "patch") { + log.Errorf("Unknown mode for range_req_handler plugin: %s", cfg.Mode) + } + log.Debugf("range_rew_handler: load success: %+v\n", cfg) + return +} + +// rangeReqHandlerOnRequest determines if there is a Range header, and puts the ranges in *d.Context as a []byteRanges +func rangeReqHandlerOnRequest(icfg interface{}, d OnRequestData) bool { + rHeader := d.R.Header.Get("Range") + if rHeader == "" { + log.Debugf("No Range header found") Review comment: `Debugf` should print a newline at the end This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188121571 ## File path: grove/plugin/range_req_handler.go ## @@ -0,0 +1,225 @@ +package plugin + +/* + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/apache/incubator-trafficcontrol/grove/web" + "github.com/apache/incubator-trafficcontrol/lib/go-log" +) + +type byteRange struct { + Start int64 + End int64 +} + +type rangeRequestConfig struct { + Mode string `json::"mode"` +} + +func init() { + AddPlugin(1, Funcs{ + load:rangeReqHandleLoad, + onRequest: rangeReqHandlerOnRequest, + beforeCacheLookUp: rangeReqHandleBeforeCacheLookup, + beforeParentRequest: rangeReqHandleBeforeParent, + beforeRespond: rangeReqHandleBeforeRespond, + }) +} + +// rangeReqHandleLoad loads the configuration +func rangeReqHandleLoad(b json.RawMessage) interface{} { + cfg := rangeRequestConfig{} + log.Errorf("rangeReqHandleLoad loading: %s", b) Review comment: `Errorf` should print a newline This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188121571 ## File path: grove/plugin/range_req_handler.go ## @@ -0,0 +1,225 @@ +package plugin + +/* + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/apache/incubator-trafficcontrol/grove/web" + "github.com/apache/incubator-trafficcontrol/lib/go-log" +) + +type byteRange struct { + Start int64 + End int64 +} + +type rangeRequestConfig struct { + Mode string `json::"mode"` +} + +func init() { + AddPlugin(1, Funcs{ + load:rangeReqHandleLoad, + onRequest: rangeReqHandlerOnRequest, + beforeCacheLookUp: rangeReqHandleBeforeCacheLookup, + beforeParentRequest: rangeReqHandleBeforeParent, + beforeRespond: rangeReqHandleBeforeRespond, + }) +} + +// rangeReqHandleLoad loads the configuration +func rangeReqHandleLoad(b json.RawMessage) interface{} { + cfg := rangeRequestConfig{} + log.Errorf("rangeReqHandleLoad loading: %s", b) Review comment: `Errorf` should print a newline. Alternatively, `log.Errorln("rangeReqHandleLoad loading: " + string(b))` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188121450 ## File path: grove/plugin/range_req_handler.go ## @@ -0,0 +1,225 @@ +package plugin + +/* + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + + "github.com/apache/incubator-trafficcontrol/grove/web" + "github.com/apache/incubator-trafficcontrol/lib/go-log" +) + +type byteRange struct { + Start int64 + End int64 +} + +type rangeRequestConfig struct { + Mode string `json::"mode"` Review comment: Typo, should be ``` `json:"mode"` ``` ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2262: Part 1 of Range Request handling
rob05c commented on a change in pull request #2262: Part 1 of Range Request handling URL: https://github.com/apache/incubator-trafficcontrol/pull/2262#discussion_r188121319 ## File path: grove/plugin/plugin.go ## @@ -91,6 +92,12 @@ type BeforeRespondData struct { Context *interface{} } +type BeforeCacheLookUpData struct { Review comment: Any objection to putting the overrideFunc in here? So it'd be `type BeforeCacheLookUpData struct { OverrideCacheKey func(string) ...` and `d.OverrideCacheKey("foo")` ? IMO it'd be cleaner and simpler, to avoid adding the extra parameter to everything. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2246: Add HitCount and FreshFor
rob05c commented on a change in pull request #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#discussion_r188120447 ## File path: grove/web/util.go ## @@ -198,3 +200,145 @@ func ParseHTTPDate(d string) (time.Time, bool) { // RemapTextKey is the plugin shared data key inserted by grovetccfg for the Remap Line of the Delivery Service in Traffic Control, Traffic Ops. const RemapTextKey = "remap_text" + +const Day = time.Hour * time.Duration(24) + +// GetHTTPDeltaSeconds is a helper function which gets an HTTP Delta Seconds from the given map (which is typically a `http.Header` or `CacheControl`. Returns false if the given key doesn't exist in the map, or if the value isn't a valid Delta Seconds per RFC2616§3.3.2. +func GetHTTPDeltaSeconds(m map[string][]string, key string) (time.Duration, bool) { Review comment: Is there a big reason these rules functions were moved? This function, and all the helper functions (`getCurrentAge`, `correctedInitialAge`, etc) are all very specifically implementing RFC 7234 and RFC 2616 rules. The `rules.go` file is intended to encapsulate those RFC rules, so someone doesn't think they can just change them to do what they need at the time. To make it more obvious "this file and these functions specifically implement the RFC". So, I'd kind of like to keep the Cache-Control RFC logic there. I've considered even giving `rules.go` its own package, for that reason. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2246: Add HitCount and FreshFor
rob05c commented on a change in pull request #2246: Add HitCount and FreshFor URL: https://github.com/apache/incubator-trafficcontrol/pull/2246#discussion_r188119204 ## File path: grove/plugin/http_cacheinspector.go ## @@ -169,16 +170,17 @@ func cacheinspect(icfg interface{}, d OnRequestData) bool { w.Write([]byte(fmt.Sprintf("showing only first %d and last %d:\n\n", head, tail))) } - w.Write([]byte(fmt.Sprintf(" #\t\tCode\tSize\tAge\t\t\tKey\n"))) + w.Write([]byte(fmt.Sprintf("#Code Size Age FreshForHitCount Key\n"))) for i, key := range keys { - if (doSearch && !strings.Contains(key, searchArr[0])) || !doSearch && (i > tail && i < len(keys)-head) { + if (doSearch && !strings.Contains(key, searchArr[0])) || !doSearch && (i >= tail && i < len(keys)-head) { continue } cacheObject, _ := d.Stats.CachePeek(key, cName) age := time.Now().Sub(cacheObject.ReqRespTime) - w.Write([]byte(fmt.Sprintf(" %05d\t%d\t%s\t%-20v\thttp://%s%s?key=%s=%s\;>%s\n", - i, cacheObject.Code, bytefmt.ByteSize(cacheObject.Size), age, req.Host, CacheStatsEndpoint, url.QueryEscape(key), cName, key))) + freshFor := web.FreshFor(cacheObject.RespHeaders, cacheObject.RespCacheControl, cacheObject.ReqRespTime, cacheObject.RespRespTime) + w.Write([]byte(fmt.Sprintf(" %8d%8d%10s%22v%22v%12d http://%s%s?key=%s=%s\;>%s\n", Review comment: I don't think you're allowed to have more symbols than letters on a line, when it's not Perl. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188116061 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -316,12 +698,123 @@ func selectQuery() string { JOIN phys_location pl ON s.phys_location = pl.id JOIN profile p ON s.profile = p.id JOIN status st ON s.status = st.id - JOIN type t ON s.type = t.id - WHERE s.id in (select server from deliveryservice_server where deliveryservice = $1)` + JOIN type t ON s.type = t.id ` return selectStmt } +type TODSSDeliveryService tc.DSSDeliveryService + +var dserviceRef = TODSSDeliveryService(tc.DSSDeliveryService{}) + +func GetDServiceRef() *TODSSDeliveryService { + return +} + +// api/1.1/servers/{id}/deliveryservices$ +func (dss *TODSSDeliveryService) Read(db *sqlx.DB, params map[string]string, user auth.CurrentUser) ([]interface{}, []error, tc.ApiErrorType) { + var err error = nil + orderby := params["orderby"] + serverId := params["id"] + + if orderby == "" { + orderby = "deliveryService" + } + + query := SDSSelectQuery() + log.Debugln("Query is ", query) + + rows, err := db.Queryx(query, serverId ) + if err != nil { + log.Errorf("Error querying DeliveryserviceServers: %v", err) + return nil, []error{tc.DBError}, tc.SystemError + } + defer rows.Close() + + services := []interface{}{} + for rows.Next() { + var s tc.DSSDeliveryService + if err = rows.StructScan(); err != nil { + log.Errorf("error parsing dss rows: %v", err) + return nil, []error{tc.DBError}, tc.SystemError + } + services = append(services, s) + } + + return services, []error{}, tc.NoError +} + +func SDSSelectQuery() string { + + const JumboFrameBPS = 9000 Review comment: Not used, should be removed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on issue #2232: Rewrite Traffic Ops CRConfig Table Endpoints in Go
rob05c commented on issue #2232: Rewrite Traffic Ops CRConfig Table Endpoints in Go URL: https://github.com/apache/incubator-trafficcontrol/issues/2232#issuecomment-387444293 Ok, we have PRs for all major table routes, but many of them are only "CRUD", and missing additional endpoints. So I'm listing all Perl API routes here. * if you've verified a route does not use one of the CRConfig tables (above), follow it with a bullet noting that. Be sure to verify any tables the route does use do not link a CRConfig table, in those tables' `traffic_ops/app/lib/Schema/Result` * if there is an unmerged PR, follow with a bullet linking the PR * if the route is in Go, in master, check it off - [x] get /api/1.1/asns - [x] get /api/1.2/asns - [x] get /api/1.1/asns/:id - [x] post /api/1.1/asns - [x] put /api/1.1/asns/:id - [x] delete /api/1.1/asns/:id - [ ] get /api/1.1/caches/stats - [x] get /api/1.1/cachegroups - [ ] get /api/1.1/cachegroups/trimmed - [x] get /api/1.1/cachegroups/:id - [x] post /api/1.1/cachegroups - [x] put /api/1.1/cachegroups/:id - [x] delete /api/1.1/cachegroups/:id - [ ] post /api/1.1/cachegroups/:id/deliveryservices - [ ] post /api/1.1/cachegroups/:id/queue_update - [x] get /api/1.1/cdns - [x] get /api/1.1/cdns/:id - [ ] get /api/1.1/cdns/name/:name - [x] post /api/1.1/cdns - [x] put /api/1.1/cdns/:id - [x] delete /api/1.1/cdns/:id - [ ] delete /api/1.1/cdns/name/:name - [ ] post /api/1.1/cdns/:id/queue_update - [ ] get /api/1.1/cdns/health - [ ] get /api/1.1/cdns/:name/health - [ ] get /api/1.1/cdns/capacity - [ ] get /api/1.1/cdns/routing - [x] get /api/1.1/cdns/:name/snapshot - [x] get /api/1.1/cdns/:name/snapshot/new - [x] put /api/1.1/cdns/:id/snapshot - [x] put /api/1.1/snapshot/:cdn_name - [ ] get /api/1.1/cdns/metric_types/:metric_type/start_date/:start_date/end_date/:end_date - [ ] get /api/1.1/cdns/name/:name/dnsseckeys - [ ] post /api/1.1/cdns/dnsseckeys/generate - [ ] get /api/1.1/cdns/name/:name/dnsseckeys/delete - [ ] get /internal/api/1.1/cdns/dnsseckeys/refresh - [ ] get /api/1.1/cdns/name/:name/sslkeys - [ ] get /api/1.1/cdns/configs - [ ] get /api/1.1/cdns/:name/configs/routing - [x] get /api/1.1/cdns/:name/configs/monitoring - [ ] get /api/1.1/cdns/domains - [ ] get /api/1.1/logs - [ ] get /api/1.1/logs/:days/days - [ ] get /api/1.1/logs/newcount - [ ] get /api/1.1/servers/:id/configfiles/ats - [ ] get /api/1.1/profiles/:id/configfiles/ats/:filename - [ ] get /api/1.1/servers/:id/configfiles/ats/:filename - [ ] get /api/1.1/cdns/:id/configfiles/ats/:filename - [ ] get /api/1.1/dbdump - [ ] get /api/1.1/deliveryservices - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [ ] get /api/1.1/deliveryservices/:id - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [ ] post /api/1.1/deliveryservices - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [ ] put /api/1.1/deliveryservices/:id - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [ ] put /api/1.1/deliveryservices/:id/safe - [ ] delete /api/1.1/deliveryservices/:id - PR https://github.com/apache/incubator-trafficcontrol/pull/2124 - [x] get /api/1.1/servers/:id/deliveryservices - [ ] post /api/1.1/deliveryservices/:xml_id/servers - [ ] delete /api/1.1/deliveryservice_server/:dsId/:serverId - [ ] get /api/1.1/deliveryservices/:id/health - [ ] get /api/1.1/deliveryservices/:id/capacity - [ ] get /api/1.1/deliveryservices/:id/routing - [ ] get /api/1.1/deliveryservices/:id/state - [ ] post /api/1.1/deliveryservices/request - [ ] get /internal/api/1.1/steering - [ ] get /internal/api/1.1/steering/:xml_id - [ ] put /internal/api/1.1/steering/:xml_id - [ ] get /api/1.1/steering/:id/targets - [ ] get /api/1.1/steering/:id/targets/:target_id - [ ] post /api/1.1/steering/:id/targets - [ ] put /api/1.1/steering/:id/targets/:target_id - [ ] delete /api/1.1/steering/:id/targets/:target_id - [x] get /api/1.1/deliveryservices/xmlId/:xmlid/sslkeys - [ ] post /api/1.1/deliveryservices/sslkeys/generate - [x] post /api/1.1/deliveryservices/sslkeys/add - [ ] get /api/1.1/deliveryservices/xmlId/:xmlid/sslkeys/delete - [ ] post /api/1.1/deliveryservices/xmlId/:xmlId/urlkeys/generate - [ ] post /api/1.1/deliveryservices/xmlId/:xmlId/urlkeys/copyFromXmlId/:copyFromXmlId - [ ] get /api/1.1/deliveryservices/xmlId/:xmlId/urlkeys - [ ] get /api/1.1/deliveryservices/:id/urlkeys - [ ] get /api/1.1/deliveryservices_regexes - PR https://github.com/apache/incubator-trafficcontrol/pull/2229 - [ ] get /api/1.1/deliveryservices/:dsId/regexes - PR https://github.com/apache/incubator-trafficcontrol/pull/2229 - [ ] get /api/1.1/deliveryservices/:dsId/regexes/:id - PR
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188118026 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -97,130 +100,107 @@ func (dss *TODeliveryServiceServer) Validate(db *sqlx.DB) []error { return tovalidate.ToErrors(errs) } -//The TODeliveryServiceServer implementation of the Creator interface -//all implementations of Creator should use transactions and return the proper errorType -//ParsePQUniqueConstraintError is used to determine if a profileparameter with conflicting values exists -//if so, it will return an errorType of DataConflict and the type should be appended to the -//generic error message returned -//The insert sql returns the profile and lastUpdated values of the newly inserted profileparameter and have -//to be added to the struct -func (dss *TODeliveryServiceServer) Create(db *sqlx.DB, user auth.CurrentUser) (error, tc.ApiErrorType) { - rollbackTransaction := true - tx, err := db.Beginx() - defer func() { - if tx == nil || !rollbackTransaction { - return +// api/1.1/deliveryserviceserver$ +func ReadDSSHandler(db *sqlx.DB) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + //create error function with ResponseWriter and Request + handleErrs := tc.GetHandleErrorsFunc(w, r) + + ctx := r.Context() + + // Load the PathParams into the query parameters for pass through + params, err := api.GetCombinedParams(r) + if err != nil { + log.Errorf("unable to get parameters from request: %s", err) + handleErrs(http.StatusInternalServerError, err) } - err := tx.Rollback() + + user, err := auth.GetCurrentUser(ctx) if err != nil { - log.Errorln(errors.New("rolling back transaction: " + err.Error())) + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return } - }() - if err != nil { - log.Error.Printf("could not begin transaction: %v", err) - return tc.DBError, tc.SystemError - } - resultRows, err := tx.NamedQuery(insertQuery(), dss) - if err != nil { - if pqErr, ok := err.(*pq.Error); ok { - err, eType := dbhelpers.ParsePQUniqueConstraintError(pqErr) - if eType == tc.DataConflictError { - return errors.New("a parameter with " + err.Error()), eType - } - return err, eType + results, errs, errType := GetRefType().readDSS(db, params, *user) + if len(errs) > 0 { + tc.HandleErrorsWithType(errs, errType, handleErrs) + return } - log.Errorf("received non pq error: %++v from create execution", err) - return tc.DBError, tc.SystemError - } - defer resultRows.Close() - - var ds_id int - var server_id int - var lastUpdated tc.TimeNoMod - rowsAffected := 0 - for resultRows.Next() { - rowsAffected++ - if err := resultRows.Scan(_id, _id, ); err != nil { - log.Error.Printf("could not scan dss from insert: %s\n", err) - return tc.DBError, tc.SystemError + respBts, err := json.Marshal(results) + if err != nil { + handleErrs(http.StatusInternalServerError, err) + return } - } - if rowsAffected == 0 { - err = errors.New("no deliveryServiceServer was inserted, nothing to return") - log.Errorln(err) - return tc.DBError, tc.SystemError - } - if rowsAffected > 1 { - err = errors.New("too many ids returned from parameter insert") - log.Errorln(err) - return tc.DBError, tc.SystemError - } - dss.SetKeys(map[string]interface{}{"deliveryservice": ds_id, "server": server_id}) - dss.LastUpdated = - err = tx.Commit() - if err != nil { - log.Errorln("Could not commit transaction: ", err) - return tc.DBError, tc.SystemError + w.Header().Set("Content-Type", "application/json") + fmt.Fprintf(w, "%s", respBts) } - rollbackTransaction = false - return nil, tc.NoError } +func (dss *TODeliveryServiceServer) readDSS(db *sqlx.DB, params map[string]string, user
[GitHub] rob05c opened a new pull request #2270: Add TO Go Servers Details endpoint
rob05c opened a new pull request #2270: Add TO Go Servers Details endpoint URL: https://github.com/apache/incubator-trafficcontrol/pull/2270 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188110723 ## File path: infrastructure/docker/traffic_ops/Dockerfile_psql ## @@ -0,0 +1,20 @@ +FROM centos/systemd Review comment: Can you pull these docker things into a separate PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188112817 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) + var xmlId string + row.Scan() + hasAccess, err, apiStatus := tenant.HasTenant(*user, xmlId, db) + if !hasAccess { + switch apiStatus { + case tc.SystemError: + handleErrs(http.StatusInternalServerError, err) + return + case tc.DataMissingError: + handleErrs(http.StatusBadRequest, err) + return + case tc.ForbiddenError: + handleErrs(http.StatusForbidden, err) + return + } + } + + // perform the insert transaction + rollbackTransaction := true + tx, err := db.Beginx() + defer func() { + if tx == nil || !rollbackTransaction { + return + } + err := tx.Rollback() + if err != nil { + log.Errorln(errors.New("rolling back transaction: " + err.Error())) + } + }() + + if err != nil { + log.Error.Printf("could not begin transaction: %v", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + if *payload.Replace { Review comment: Panic, need to check for nil before dereferencing. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188114096 ## File path: infrastructure/docker/traffic_ops/dbInit.sh ## @@ -0,0 +1,8 @@ +#!/usr/bin/env bash Review comment: This file needs an Apache license header This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188117012 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) + var xmlId string + row.Scan() + hasAccess, err, apiStatus := tenant.HasTenant(*user, xmlId, db) + if !hasAccess { + switch apiStatus { + case tc.SystemError: + handleErrs(http.StatusInternalServerError, err) + return + case tc.DataMissingError: + handleErrs(http.StatusBadRequest, err) + return + case tc.ForbiddenError: + handleErrs(http.StatusForbidden, err) + return + } + } + + // perform the insert transaction + rollbackTransaction := true + tx, err := db.Beginx() + defer func() { + if tx == nil || !rollbackTransaction { + return + } + err := tx.Rollback() + if err != nil { + log.Errorln(errors.New("rolling back transaction: " + err.Error())) + } + }() + + if err != nil { + log.Error.Printf("could not begin transaction: %v", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + if *payload.Replace { + // delete existing + rows, err := db.Queryx( "DELETE FROM deliveryservice_server WHERE deliveryservice = $1", *dsId) + if err != nil { + log.Errorf("unable to remove the existing servers assigned to the delivery service: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + defer rows.Close() + } + + i := 0 + respServers := []int{} + + for i < len(servers) { + dtos := map[string]interface{}{"id":dsId, "server":servers[i]} +
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188112552 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) Review comment: This will panic if the user input has no `dsId` field. Needs to check for `nil` before dereferencing. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188113319 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) + var xmlId string + row.Scan() + hasAccess, err, apiStatus := tenant.HasTenant(*user, xmlId, db) + if !hasAccess { + switch apiStatus { + case tc.SystemError: + handleErrs(http.StatusInternalServerError, err) + return + case tc.DataMissingError: + handleErrs(http.StatusBadRequest, err) + return + case tc.ForbiddenError: + handleErrs(http.StatusForbidden, err) + return + } + } + + // perform the insert transaction + rollbackTransaction := true + tx, err := db.Beginx() + defer func() { + if tx == nil || !rollbackTransaction { + return + } + err := tx.Rollback() + if err != nil { + log.Errorln(errors.New("rolling back transaction: " + err.Error())) + } + }() + + if err != nil { + log.Error.Printf("could not begin transaction: %v", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + if *payload.Replace { + // delete existing + rows, err := db.Queryx( "DELETE FROM deliveryservice_server WHERE deliveryservice = $1", *dsId) + if err != nil { + log.Errorf("unable to remove the existing servers assigned to the delivery service: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + defer rows.Close() + } + + i := 0 + respServers := []int{} + + for i < len(servers) { Review comment: This will be clearer and safer with a range loop: `for _, server := range servers {`
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188111222 ## File path: lib/go-tc/deliveryservice_servers.go ## @@ -78,3 +99,65 @@ type DssServer struct { TypeID *int `json:"typeId" db:"server_type_id"` UpdPending *bool`json:"updPending" db:"upd_pending"` } + +// DeliveryServiceNullable - a version of the deliveryservice that allows for all fields to be null Review comment: type-level comment should start with `// DSSDeliveryService` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188114148 ## File path: infrastructure/docker/traffic_ops/Dockerfile_psql ## @@ -0,0 +1,20 @@ +FROM centos/systemd Review comment: File needs an Apache license header This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188111742 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -97,130 +100,107 @@ func (dss *TODeliveryServiceServer) Validate(db *sqlx.DB) []error { return tovalidate.ToErrors(errs) } -//The TODeliveryServiceServer implementation of the Creator interface -//all implementations of Creator should use transactions and return the proper errorType -//ParsePQUniqueConstraintError is used to determine if a profileparameter with conflicting values exists -//if so, it will return an errorType of DataConflict and the type should be appended to the -//generic error message returned -//The insert sql returns the profile and lastUpdated values of the newly inserted profileparameter and have -//to be added to the struct -func (dss *TODeliveryServiceServer) Create(db *sqlx.DB, user auth.CurrentUser) (error, tc.ApiErrorType) { - rollbackTransaction := true - tx, err := db.Beginx() - defer func() { - if tx == nil || !rollbackTransaction { - return +// api/1.1/deliveryserviceserver$ +func ReadDSSHandler(db *sqlx.DB) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + //create error function with ResponseWriter and Request + handleErrs := tc.GetHandleErrorsFunc(w, r) + + ctx := r.Context() + + // Load the PathParams into the query parameters for pass through + params, err := api.GetCombinedParams(r) + if err != nil { + log.Errorf("unable to get parameters from request: %s", err) + handleErrs(http.StatusInternalServerError, err) } - err := tx.Rollback() + + user, err := auth.GetCurrentUser(ctx) if err != nil { - log.Errorln(errors.New("rolling back transaction: " + err.Error())) + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return } - }() - if err != nil { - log.Error.Printf("could not begin transaction: %v", err) - return tc.DBError, tc.SystemError - } - resultRows, err := tx.NamedQuery(insertQuery(), dss) - if err != nil { - if pqErr, ok := err.(*pq.Error); ok { - err, eType := dbhelpers.ParsePQUniqueConstraintError(pqErr) - if eType == tc.DataConflictError { - return errors.New("a parameter with " + err.Error()), eType - } - return err, eType + results, errs, errType := GetRefType().readDSS(db, params, *user) + if len(errs) > 0 { + tc.HandleErrorsWithType(errs, errType, handleErrs) + return } - log.Errorf("received non pq error: %++v from create execution", err) - return tc.DBError, tc.SystemError - } - defer resultRows.Close() - - var ds_id int - var server_id int - var lastUpdated tc.TimeNoMod - rowsAffected := 0 - for resultRows.Next() { - rowsAffected++ - if err := resultRows.Scan(_id, _id, ); err != nil { - log.Error.Printf("could not scan dss from insert: %s\n", err) - return tc.DBError, tc.SystemError + respBts, err := json.Marshal(results) + if err != nil { + handleErrs(http.StatusInternalServerError, err) + return } - } - if rowsAffected == 0 { - err = errors.New("no deliveryServiceServer was inserted, nothing to return") - log.Errorln(err) - return tc.DBError, tc.SystemError - } - if rowsAffected > 1 { - err = errors.New("too many ids returned from parameter insert") - log.Errorln(err) - return tc.DBError, tc.SystemError - } - dss.SetKeys(map[string]interface{}{"deliveryservice": ds_id, "server": server_id}) - dss.LastUpdated = - err = tx.Commit() - if err != nil { - log.Errorln("Could not commit transaction: ", err) - return tc.DBError, tc.SystemError + w.Header().Set("Content-Type", "application/json") + fmt.Fprintf(w, "%s", respBts) } - rollbackTransaction = false - return nil, tc.NoError } +func (dss *TODeliveryServiceServer) readDSS(db *sqlx.DB, params map[string]string, user
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188115566 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) + var xmlId string + row.Scan() + hasAccess, err, apiStatus := tenant.HasTenant(*user, xmlId, db) + if !hasAccess { + switch apiStatus { + case tc.SystemError: + handleErrs(http.StatusInternalServerError, err) + return + case tc.DataMissingError: + handleErrs(http.StatusBadRequest, err) + return + case tc.ForbiddenError: + handleErrs(http.StatusForbidden, err) + return + } + } + + // perform the insert transaction + rollbackTransaction := true + tx, err := db.Beginx() + defer func() { + if tx == nil || !rollbackTransaction { + return + } + err := tx.Rollback() + if err != nil { + log.Errorln(errors.New("rolling back transaction: " + err.Error())) + } + }() + + if err != nil { + log.Error.Printf("could not begin transaction: %v", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + if *payload.Replace { + // delete existing + rows, err := db.Queryx( "DELETE FROM deliveryservice_server WHERE deliveryservice = $1", *dsId) + if err != nil { + log.Errorf("unable to remove the existing servers assigned to the delivery service: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + defer rows.Close() + } + + i := 0 + respServers := []int{} + + for i < len(servers) { + dtos := map[string]interface{}{"id":dsId, "server":servers[i]} +
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188114240 ## File path: lib/go-tc/deliveryservice_servers.go ## @@ -1,6 +1,8 @@ package tc -import "time" +import ( Review comment: File needs an Apache license header This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188110995 ## File path: lib/go-tc/deliveryservice_servers.go ## @@ -19,21 +21,40 @@ import "time" // DeliveryServiceServerResponse ... type DeliveryServiceServerResponse struct { + Orderby string `json:"orderby"` Response []DeliveryServiceServer `json:"response"` Size int `json:"size"` - OrderBy string `json:"orderby"` Limitint `json:"limit"` } +type DSSMapResponse struct { + DsIdint `json:"dsId"` Review comment: This needs `gofmt` run on it. You can quickly run it on all the files in a directory via `gofmt -w ./...` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188114503 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -97,130 +100,107 @@ func (dss *TODeliveryServiceServer) Validate(db *sqlx.DB) []error { return tovalidate.ToErrors(errs) } -//The TODeliveryServiceServer implementation of the Creator interface -//all implementations of Creator should use transactions and return the proper errorType -//ParsePQUniqueConstraintError is used to determine if a profileparameter with conflicting values exists -//if so, it will return an errorType of DataConflict and the type should be appended to the -//generic error message returned -//The insert sql returns the profile and lastUpdated values of the newly inserted profileparameter and have -//to be added to the struct -func (dss *TODeliveryServiceServer) Create(db *sqlx.DB, user auth.CurrentUser) (error, tc.ApiErrorType) { - rollbackTransaction := true - tx, err := db.Beginx() - defer func() { - if tx == nil || !rollbackTransaction { - return +// api/1.1/deliveryserviceserver$ +func ReadDSSHandler(db *sqlx.DB) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + //create error function with ResponseWriter and Request + handleErrs := tc.GetHandleErrorsFunc(w, r) + + ctx := r.Context() + + // Load the PathParams into the query parameters for pass through + params, err := api.GetCombinedParams(r) + if err != nil { + log.Errorf("unable to get parameters from request: %s", err) + handleErrs(http.StatusInternalServerError, err) } - err := tx.Rollback() + + user, err := auth.GetCurrentUser(ctx) if err != nil { - log.Errorln(errors.New("rolling back transaction: " + err.Error())) + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return } - }() - if err != nil { - log.Error.Printf("could not begin transaction: %v", err) - return tc.DBError, tc.SystemError - } - resultRows, err := tx.NamedQuery(insertQuery(), dss) - if err != nil { - if pqErr, ok := err.(*pq.Error); ok { - err, eType := dbhelpers.ParsePQUniqueConstraintError(pqErr) - if eType == tc.DataConflictError { - return errors.New("a parameter with " + err.Error()), eType - } - return err, eType + results, errs, errType := GetRefType().readDSS(db, params, *user) + if len(errs) > 0 { + tc.HandleErrorsWithType(errs, errType, handleErrs) + return } - log.Errorf("received non pq error: %++v from create execution", err) - return tc.DBError, tc.SystemError - } - defer resultRows.Close() - - var ds_id int - var server_id int - var lastUpdated tc.TimeNoMod - rowsAffected := 0 - for resultRows.Next() { - rowsAffected++ - if err := resultRows.Scan(_id, _id, ); err != nil { - log.Error.Printf("could not scan dss from insert: %s\n", err) - return tc.DBError, tc.SystemError + respBts, err := json.Marshal(results) + if err != nil { + handleErrs(http.StatusInternalServerError, err) + return } - } - if rowsAffected == 0 { - err = errors.New("no deliveryServiceServer was inserted, nothing to return") - log.Errorln(err) - return tc.DBError, tc.SystemError - } - if rowsAffected > 1 { - err = errors.New("too many ids returned from parameter insert") - log.Errorln(err) - return tc.DBError, tc.SystemError - } - dss.SetKeys(map[string]interface{}{"deliveryservice": ds_id, "server": server_id}) - dss.LastUpdated = - err = tx.Commit() - if err != nil { - log.Errorln("Could not commit transaction: ", err) - return tc.DBError, tc.SystemError + w.Header().Set("Content-Type", "application/json") + fmt.Fprintf(w, "%s", respBts) } - rollbackTransaction = false - return nil, tc.NoError } +func (dss *TODeliveryServiceServer) readDSS(db *sqlx.DB, params map[string]string, user
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188112182 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) Review comment: This is leaking the body, if there's an error. Need to move `defer r.Body.Close()` to the top of the function. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188114777 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) + var xmlId string + row.Scan() + hasAccess, err, apiStatus := tenant.HasTenant(*user, xmlId, db) + if !hasAccess { + switch apiStatus { + case tc.SystemError: + handleErrs(http.StatusInternalServerError, err) + return + case tc.DataMissingError: + handleErrs(http.StatusBadRequest, err) + return + case tc.ForbiddenError: + handleErrs(http.StatusForbidden, err) + return + } + } + + // perform the insert transaction + rollbackTransaction := true + tx, err := db.Beginx() + defer func() { + if tx == nil || !rollbackTransaction { + return + } + err := tx.Rollback() + if err != nil { + log.Errorln(errors.New("rolling back transaction: " + err.Error())) + } + }() + + if err != nil { + log.Error.Printf("could not begin transaction: %v", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + if *payload.Replace { + // delete existing + rows, err := db.Queryx( "DELETE FROM deliveryservice_server WHERE deliveryservice = $1", *dsId) + if err != nil { + log.Errorf("unable to remove the existing servers assigned to the delivery service: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + defer rows.Close() + } + + i := 0 + respServers := []int{} + + for i < len(servers) { + dtos := map[string]interface{}{"id":dsId, "server":servers[i]} +
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188110995 ## File path: lib/go-tc/deliveryservice_servers.go ## @@ -19,21 +21,40 @@ import "time" // DeliveryServiceServerResponse ... type DeliveryServiceServerResponse struct { + Orderby string `json:"orderby"` Response []DeliveryServiceServer `json:"response"` Size int `json:"size"` - OrderBy string `json:"orderby"` Limitint `json:"limit"` } +type DSSMapResponse struct { + DsIdint `json:"dsId"` Review comment: This needs `gofmt` run on it. You can quickly run it on all the files in a directory via via `gofmt -w ./...` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188115772 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) + var xmlId string + row.Scan() + hasAccess, err, apiStatus := tenant.HasTenant(*user, xmlId, db) + if !hasAccess { + switch apiStatus { + case tc.SystemError: + handleErrs(http.StatusInternalServerError, err) + return + case tc.DataMissingError: + handleErrs(http.StatusBadRequest, err) + return + case tc.ForbiddenError: + handleErrs(http.StatusForbidden, err) + return + } + } + + // perform the insert transaction + rollbackTransaction := true + tx, err := db.Beginx() + defer func() { + if tx == nil || !rollbackTransaction { + return + } + err := tx.Rollback() + if err != nil { + log.Errorln(errors.New("rolling back transaction: " + err.Error())) + } + }() + + if err != nil { + log.Error.Printf("could not begin transaction: %v", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + if *payload.Replace { + // delete existing + rows, err := db.Queryx( "DELETE FROM deliveryservice_server WHERE deliveryservice = $1", *dsId) + if err != nil { + log.Errorf("unable to remove the existing servers assigned to the delivery service: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + defer rows.Close() + } + + i := 0 + respServers := []int{} + + for i < len(servers) { + dtos := map[string]interface{}{"id":dsId, "server":servers[i]} +
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188114875 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) + var xmlId string + row.Scan() + hasAccess, err, apiStatus := tenant.HasTenant(*user, xmlId, db) + if !hasAccess { + switch apiStatus { + case tc.SystemError: + handleErrs(http.StatusInternalServerError, err) + return + case tc.DataMissingError: + handleErrs(http.StatusBadRequest, err) + return + case tc.ForbiddenError: + handleErrs(http.StatusForbidden, err) + return + } + } + + // perform the insert transaction + rollbackTransaction := true + tx, err := db.Beginx() + defer func() { + if tx == nil || !rollbackTransaction { + return + } + err := tx.Rollback() + if err != nil { + log.Errorln(errors.New("rolling back transaction: " + err.Error())) + } + }() + + if err != nil { + log.Error.Printf("could not begin transaction: %v", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + if *payload.Replace { + // delete existing + rows, err := db.Queryx( "DELETE FROM deliveryservice_server WHERE deliveryservice = $1", *dsId) + if err != nil { + log.Errorf("unable to remove the existing servers assigned to the delivery service: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + defer rows.Close() + } + + i := 0 + respServers := []int{} + + for i < len(servers) { + dtos := map[string]interface{}{"id":dsId, "server":servers[i]} +
[GitHub] rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go
rob05c commented on a change in pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#discussion_r188115110 ## File path: traffic_ops/traffic_ops_golang/deliveryservice/servers/servers.go ## @@ -263,7 +243,409 @@ func (dss *TODeliveryServiceServer) Delete(db *sqlx.DB, user auth.CurrentUser) ( rollbackTransaction = false return nil, tc.NoError } -func selectQuery() string { + +func selectQuery( orderby string, limit string, offset string) string { + + selectStmt := `SELECT + s.deliveryService, + s.server, + s.last_updated + FROM deliveryservice_server s + ORDER BY `+ orderby +` LIMIT `+limit+` OFFSET `+offset+` ROWS` + + return selectStmt +} + +func deleteQuery() string { + query := `DELETE FROM deliveryservice_server + WHERE deliveryservice=:deliveryservice and server=:server` + return query +} + + +type DSServers struct { + DsId *int `json:"dsId" db:"deliveryservice"` + Servers []int `json:"servers"` + Replace *bool `json:"replace"` +} + +type TODSServers DSServers +var dsserversRef = TODSServers(DSServers{}) + +func GetServersForDsIdRef() *TODSServers { + return +} + +func GetReplaceHandler( db *sqlx.DB ) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handleErrs := tc.GetHandleErrorsFunc(w, r) + ctx := r.Context() + user, err := auth.GetCurrentUser(ctx) + if err != nil { + log.Errorf("unable to retrieve current user from context: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // get list of server Ids to insert + defer r.Body.Close() + payload := GetServersForDsIdRef() + servers := payload.Servers + dsId:= payload.DsId + + if err := json.NewDecoder(r.Body).Decode(payload); err != nil { + log.Errorf("Error trying to decode the request body: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + // if the object has tenancy enabled, check that user is able to access the tenant + // check user tenancy access to this resource. + row := db.QueryRow("SELECT xml_id FROM deliveryservice WHERE id = $1", *dsId) + var xmlId string + row.Scan() + hasAccess, err, apiStatus := tenant.HasTenant(*user, xmlId, db) + if !hasAccess { + switch apiStatus { + case tc.SystemError: + handleErrs(http.StatusInternalServerError, err) + return + case tc.DataMissingError: + handleErrs(http.StatusBadRequest, err) + return + case tc.ForbiddenError: + handleErrs(http.StatusForbidden, err) + return + } + } + + // perform the insert transaction + rollbackTransaction := true + tx, err := db.Beginx() + defer func() { + if tx == nil || !rollbackTransaction { + return + } + err := tx.Rollback() + if err != nil { + log.Errorln(errors.New("rolling back transaction: " + err.Error())) + } + }() + + if err != nil { + log.Error.Printf("could not begin transaction: %v", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + if *payload.Replace { + // delete existing + rows, err := db.Queryx( "DELETE FROM deliveryservice_server WHERE deliveryservice = $1", *dsId) + if err != nil { + log.Errorf("unable to remove the existing servers assigned to the delivery service: %s", err) + handleErrs(http.StatusInternalServerError, err) + return + } + + defer rows.Close() + } + + i := 0 + respServers := []int{} + + for i < len(servers) { + dtos := map[string]interface{}{"id":dsId, "server":servers[i]} +
[GitHub] asfgit commented on issue #2149: TO and TODB in docker front-to-back
asfgit commented on issue #2149: TO and TODB in docker front-to-back URL: https://github.com/apache/incubator-trafficcontrol/pull/2149#issuecomment-388982939 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR/1567/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2269: Deliveryservice_Server API conversion to Go
asfgit commented on issue #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#issuecomment-388975850 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #2269: Deliveryservice_Server API conversion to Go
asfgit commented on issue #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269#issuecomment-388975605 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ajschmidt opened a new pull request #2269: Deliveryservice_Server API conversion to Go
ajschmidt opened a new pull request #2269: Deliveryservice_Server API conversion to Go URL: https://github.com/apache/incubator-trafficcontrol/pull/2269 This PR converts these API methods to Go: GETapi/1.1/deliveryserviceserver POST api/1.1/deliveryserviceserver POST api/1.1/deliveryservices/{xml_id}/servers GET. api/1.1/deliveryservices/{id}/servers GETapi/1.1/deliveryservices/{id}/unassigned_servers I also added a Docker Compose file to: infrastructure/docker/traffic_ops which will quickly create a local traffic_ops and postgres Dev environment for quickly testing changes to the traffic_ops_golang proxy. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dangogh commented on issue #2149: TO and TODB in docker front-to-back
dangogh commented on issue #2149: TO and TODB in docker front-to-back URL: https://github.com/apache/incubator-trafficcontrol/pull/2149#issuecomment-388970888 closing for now while still WIP This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services