[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user asfgit commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-trafficcontrol-PR-trafficops-test/24/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user asfgit commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Carried forward from slack for reference: Password format changed from SHA1 to scrypt. http://search.cpan.org/~mik/Crypt-ScryptKDF-0.006/lib/Crypt/ScryptKDF.pm https://github.com/apache/incubator-trafficcontrol/commit/3dd1ad0c84adbebe6b3b514e61658c4eac648d50 GitHub Change TO password hashing to scrypt · apache/incubator-trafficcontrol@3dd1ad0 incubator-trafficcontrol - Mirror of Apache Traffic Control (incubating) https://godoc.org/golang.org/x/crypto/scrypt godoc.org Package scrypt Package scrypt implements the scrypt key derivation function as defined in Colin Percival's paper "Stronger Key Derivation via Sequential Memory-Hard Functions" (http://www.tarsnap.com/scrypt/scrypt.pdf). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @amiryesh I think I discovered another issue. When you send the Mojo cookie now, it flows through to Traffic Ops accordingly, but should we also need the "capabilities" guard against that backend as well. I think you've built the capabilities for only JWT and the "Claims". You need to authorize "all" routes against capabilities, not just JWT routes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @amiryesh Another useful piece of information would be timings. For example: "How long did it take to call that backend, in milliseconds?" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Another something to play with: https://github.com/elazarl/goproxy --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @amiryesh Another feature request Out of curiousity, I configured the gateway to "front" Traffic Ops entirely (and it worked!), but I had to reconfigure the "/login" (which goes to the auth.go) to point to "/jwt_token". Traffic Ops also has a route called "/login", so the mapping had a conflict. Can we build an option to allow for say a prefix mapping such as /traffic_ops so that everytime I request /traffic_ops it removes that from the context before forwarding it to an actual Traffic Ops. This will allow the gateway namespacing to prevent any potential url conflicts. Apache webserver has this feature if you want to research more: Reverse Proxy ProxyPass "/foo" "http://foo.example.com/bar; ProxyPassReverse "/foo" "http://foo.example.com/bar; So the client requests: http://reverse_proxy/foo/bar and the mapping goes to "http://foo.example.com/bar; --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @amiryesh Another request, please make the server.key, etc configurable, ops may want to put them in a different location than the current directory. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Just a thought, it might be useful to expose a route on the webfront that lists the "backend" routes it's mapping (in .json form). This way, I can remotely ask the gateway about it's "registry", which could be useful for testing. Of course, we'll have to keep security in mind, but I'm not sure that will be an issue if you've been "authenticated". --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Re: grouping auth and webfront into a single directory. Maybe this is the start of a Dev list discussion about how to structure our Microservices, where webfront and auth are apart of that structure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @amiryesh Nevermind about the 502 bad gateway. I can't seem to reproduce it now if I find it again, I'll let you know. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user amiryesh commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @dewrich Hi, Thanks for your comments. Regarding better error handling, I my tests I do get a 502 response in this case. Can you share your setup details to help me reconstruct this issue? Also, please attach console output of webfront.go. WRT grouping auth and webfront into a single directory, I am not sure about that. The API GW is the webfront server. The auth server works in conjunction with the API GW in the new architecture, but it is not part of it. I will add this explanation at the top of auth/readme, including an href to webfront's readme, hoping to make this clear. Thanks /amiry --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user amiryesh commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @limited Right --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user limited commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Thanks Amir- Perfect sense! Just to make sure I understood correctly, we will keep both the auth server and API gateway as optional components that could be switched out for production deployments if desired? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user amiryesh commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @limited Hi, I agree with your view on the benefits of using industry standard software wherever applicable. Let me share my view on how the API GW and the auth server are to be considered. IMO, the auth server and API GW implementation is to be considered as an entry point towards breaking the TO monolith into microservice architecture. The auth server is far from being production grade. It does not support standard OAuth2 flows. It probably has security flaws. More than that, users management and access control is implemented as a part of TC. I believe that in production, one would want to use different identity services to manage her users, and implement more complex authentication flows. The API GW architecture allows you to manage your users externally and provide a JWT with proper TO claims. However, this is a very complex setup. We would want users to be able to run a simple working TC instance as easy as possible. This is why the auth server was implemented. The same rational applies also to the API GW implementation. I agree that if we can configure nginx to do what we need it is most likely a better fit for production. What we do get from the API GW is custom authorization code. We have our own simple code that can verify our custom claims. I don't have much nginx experience, I think that custom authorization logic is likely to be harder to configure, but maybe I'm wrong. Hope that makes sense, /amiry --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Suggestions: 1. Documentation: Once I understood how the pieces fit together, it was relatively simple (but I also had some background about how it was built). This can be improved down the road once this gets approved and tested more 2. apigateway subdirectory I would combine the webfront and auth dirs into an "apigateway" subdir just to show they are grouped which will also help with potentially understanding how auth and webfront fit together. 3. Logging: Each request should be written to "access.log" and follow similar layouts (such as Apache Webserver), because that can be parsed and loaded for Analytics. It would also be nice to have an "apigw.log" (or something) where I can turn up logging to see more output as to how the rules are being evaluated for troubleshooting/development more rules). In case I fat fingered some syntax 4. Better Error handling: As a POC, I stood up 1 TO API and a basic microservice. Each were accessible through the gateway. When I brought down the microservice endpoint, the Gateway responded with a "blank" response. With standard servers a 502 "Bad Gateway" response is sent back to the client like this: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/502 What I liked: 1. Setup was relatively easy 2. I liked the "hot" loading of the .json rules I will continue testing, but just wanted to give initial feedback. All in all nice work! Thanks, -Dewayne --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 FYI: I was able to access a TO API endpoint from the webfront with success. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user dewrich commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @amiryesh Thanks for clarification. It wasn't obvious that the "auth" dir and "webfront" was included in the "apigateway" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user limited commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 This look pretty nice Amir- Have you considered using an open source proxy like nginx or varnish in place of writing our own? Both pieces of software can almost certainly validate JWT and perform remaps. They have the advantage of already providing support for a wide range of HTTP transformations that webfront doesn't yet have. For example, I might like to use this proxy in front of grafana to perform authentication (based on setting some HTTP request headers). With nginx or varnish, I could easily add those headers. I'm sure we'll have many other use cases arise where we'd want to continously add more functionality to the API GW proxy. Also, those other open source proxies are much better tested and have many more miles on their tires- they could be a quick path to a very stable component here. Are there certain reasons why we prefer writing our own proxy to using one of several alternatives? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user amiryesh commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Hi, glad to hear you are taking the effort of using the GW. Did you get to see webfront/README.md or auth/README.md? The auth server is responsible for login. The auth readme explains how to obtain a token, which I understand that you did. The webfront server is the API GW. The webfront readme shows token usage example: $ curl --insecure -H'Authorization: Bearer ' -Lkvs https://localhost:8080/ds/ At phase 0, auth server access tm_users, user_role and role_capabilities DB tables to compose a JWT with the user's capabilities. However, all webfront configuration is hard coded. The forwarding rules are hard coded in ruls.json and the authorization data (route-to-capability mapping) is hard coded in routes.json. Please check your routes config, and make sure your API calls are done against the webfront server and not the auth server. If you are still having problems, please attach your query samples + console output from both webfront.go and auth.go --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user ryandurfey commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Working to get a committer to evaluate. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user rob05c commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 Looks good to me (once its rebased). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-trafficcontrol issue #567: API GW phase 0 (replaces #551, depends ...
Github user mitchell852 commented on the issue: https://github.com/apache/incubator-trafficcontrol/pull/567 @amiryesh - can you rebase this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
