Re: Steering Target Geo-Ordering

[Rawlin Peters]

So I've done some more thinking about client-steering geo-ordering,
and I think the geo-sorting part should be summarized as follows:
Sort the geo-steering targets ascending by the total distance from
[client -> edge -> origin]. For targets where the total distance is
equal, prefer the target with the shortest [client -> edge] distance.

This is a more difficult task than just sorting by distance between
the client and origin, but it takes into account the fact that the
most optimal cachegroups might not be available.

- Rawlin



On Wed, Feb 28, 2018 at 11:02 AM, Rawlin Peters  wrote:
> No problem, Eric. I appreciate a good brainstorming. Really the
> Seattle-Denver-Boston example I gave was just to illustrate the idea
> of the feature; I'm not sure we'd actually go with a Geo Steering DS
> architecture that dispersed.
>
> Steering DSes aren't actually assignable to specific cachegroups,
> because the assignments of the target DSes are what's actually used in
> order to find a cache that supports the target DS. As long as the
> target DSes are assigned to all cache groups, the client will be
> routed to an optimal edge cache. If the targets are geo-ordered, then
> the client's first option is to request a target DS with an Origin
> that is also optimal to that edge cache.
>
> I think I might've confused the way I worded the problem initially.
> The target DSes should always be ordered by distance between the
> routed-to cachegroup and the Origin. That way a client in Seattle that
> was routed to an edge in Boston (if the Seattle and Denver CGs were
> down) would first request an Origin in Boston not Seattle. That way
> traffic wouldn't be be crossing the country twice.
>
> Does that make more sense?
>
> - Rawlin
>
> On Wed, Feb 28, 2018 at 7:39 AM, Eric Friedrich (efriedri)
>  wrote:
>> Thanks Rawlin-
>>   Thanks for bearing with me. I’m thinking through this and wanted to 
>> brainstorm a little.
>>
>> What’s the benefit of going to the Boston edge cache to get the Seattle or 
>> Denver DeliveryServices?
>>   - If you already have redundant origins in Boston, you’re protected 
>> against failure of a single origin.
>>   - If both Boston origins fail, the client would be routed to Denver and 
>> hit the local Denver origins.
>>   - If both Boston origins are down, then you wouldn’t be able to get Boston 
>> content in Denver anyway
>>  - But clients in Boston would now need to hit the Denver edges, perhaps 
>> overloading that CG. If this is a failure case you want to address, maybe we 
>> could do something like the following:
>>
>> Would it be possible to create multiple levels of steering delivery services?
>> - Top Level: Client Steering DS 
>> (tr.nbc.cdn.customer.com) - assigned to all CG. Does 
>> GeoOrdering based on client proximity to assigned cache groups of the target 
>> DS'
>> - Target 1: Boston Client Steering DS 
>> (tr.bos-nbc.cdn.customer.com) - 
>> assigned only to Boston CG. Ranked steering policy
>>   - Target 1.1: Boston Live DS 
>> (tr.bos-bos-nbc.cdn.customer.com), 
>> rank=1
>>   - Target 1.2: Denver Live DS 
>> (tr.den-bos-nbc.cdn.customer.com), 
>> rank=2
>>   - Target 1.3: Seattle Live DS 
>> (tr.sea-bos-nbc.cdn.customer.com), 
>> rank=3
>>
>> - Target 2: Denver Client Steering DS 
>> (tr.den-nbc.cdn.customer.com) - 
>> assigned only to Denver CG
>> - Target 3: Seattle Client Steering DS 
>> (tr.seat-nbc.cdn.customer.com) - 
>> assigned only to Seattle CG
>>
>> The goal would be this order of DS’s returned to the client
>> Location: [tr.bos-bos-nbc, tr.den-bos-nbc, tr.sea-bos-nbc, tr.den-den-nbc, 
>> tr.sea-den-nbc, r.bos-den-nbc,…. ]
>>
>> Is this what are you ultimately looking for?
>>
>> This lets us choose CG and DS based on the location of the client relative 
>> to the edge cache, rather than proximity of client to origin. Since the 
>> client is talking to the edge cache and not the origin, this seems like a 
>> better metric. Being able to compose Steering DS would also give us more 
>> flexibility for other use cases in the future as well
>>
>>
>>
>>
>> On Feb 27, 2018, at 4:40 PM, Rawlin Peters 
>> > wrote:
>>
>> Hey Eric,
>>
>> In this example I'd think that all the target DSes would need to be
>> assigned to all 3 Cache Groups. That way the client could possibly use
>> any of the target DSes from the cachegroup they're routed to. But I
>> suppose that could change on a case-by-case basis where maybe the
>> target DSes are only assigned to cache groups that are close to their
>> origins. In that case I'd think a client in Seattle would possibly be
>> routed to an edge 

Traffic Ops Access Control v2

[Jeremy Mitchell]

There has been some discussion for quite some time regarding an overhaul of
the TO access control model. I'd like to refresh eveyone's memory on that
discussion.


*Current system:*

Since the beginning, resources (or routes (UI and API)) have been locked
down by role, or more specifically, privilege level. For example, if a
route requires a privilege level of 20, then only users with the operations
role (priv level=20) or admin role (priv level=30) could access that route.
Here are our current roles (and their priv levels):

Admin (30)
Operations (20)
Portal (15)
Federation (15)
Steering (15)
ORT (11)
Read-Only (10)
Disallowed (0)

This method has served us well for quite a while but there are some
drawbacks to this approach. Here are a few I can think of:

- No clear understanding of which routes each role provides access to. For
example, what is the difference between the Admin and Operations role? All
I know is that the admin role has a priv level of 30 and operations has 20.
I can't tell you which routes an admin has access to that operations does
not without reading the code or going thru all the docs. Ain't nobody got
time for that!

- The "Additive" nature of the roles (via priv level) prevents the ability
to create unrelated roles. You can't create 2 roles with unique access.
Higher level roles always inherit from lower level roles. The Federation
role is a good example. Federation users only need access to a couple
routes yet since it has a priv-level=15, federation users look like they
can do federation, steering, portal, ort and read-only things...

- Not easy to alter the access level of a role. For example, if you wanted
the Portal role to have access to a few more routes, what would you do?
Raise priv level to 18? Not sure what that would do...if anything. You'd
have to make code changes to ensure an 18 would actually do something.

- Many API consumers have elevated permissions when they only need access
to a few routes. I.e. traffic monitors, traffic routers, traffic stats all
have to be given the admin role. so basically, they've been granted access
to do EVERYTHING when they only access a few routes.

- There is also inconsistency in how roles are enforced. Most routes use
priv level to determine access but some routes simply check if the user has
the role (i.e. steering).


*New proposed system:*

*Tenancy*

Last summer tenancy was introduced (thanks Qwilt) giving us the ability to
"scope" certain resources (delivery services, users and also tenants) to
certain users. This was a big step towards self-service as we can now limit
what certain users see. Access control is now role + tenancy (if tenancy is
applicable and turned on via the use_tenancy parameter).

*Roles/Capabilities*

Actually, a lot of work has already been done (thanks again, Qwilt) in this
area but of course, there is more to do. Let me explain a bit how it works
conceptually.

Proposed Roles:

Admin
Operations
Content Provider (formerly known as Portal)
Federation
Cache (formerly known as ORT)
Monitor (new)
Router (new)
Stats (new)
Read-Only
Disallowed

Concept:

- a user has one role
- a role has N capabilities (i.e. ds-read, ds-write, etc)
- a capability is mapped to N API endpoints (i.e. ds-read is mapped to GET
/api/deliveryservices and GET /api/deliveryservices/:id)

A user's capabilities (and not privilege level) determine whether a user
has access to an API endpoint or not.

Advantages:

- By mapping roles to capabilities and capabilities to API endpoints, it's
easy to see what level of API access each role provides. For example, easy
to see the difference between the Admin and Operations role.

- Roles are not "additive". Unrelated, unique roles can be created. For
example, the Federation role and Content Provider role (formerly Portal
role) can provide 2 completely different levels of access control.
Currently, they provide the exact same level of access because of priv
level.

- Tightly defined roles with specific capabilities provides better
security. I.e. you don't have to give a user an admin role so they can do
only 2 things.

- Can create custom roles on the fly to only include access to certain API
endpoints. If you want to create a Bob role with just the ds-read
capability, go for it. You can get very creative with your roles if you
want to. Or you can just use those that are provided.

Disadvantages:

- More setup required. All API endpoints need to be grouped into
capabilities (again, Qwilt did a lot of work in this area). Capabilities
need to be added to the appropriate roles.

If you haven't read enough at this point and are thirsty for more. There is
more here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68715910

Thanks for reading. Looking forward to your comments/concerns.

Jeremy

Re: [VOTE] Release Apache Traffic Control (incubating) 2.2.0-RC2

[Dan Kirkwood]

Hank, fyi -- from the top level of trafficcontrol, `./pkg -v` will build
all components.

-dan

On Thu, Mar 8, 2018 at 6:34 AM, Hank Beatty  wrote:

> +1
>
> Lab Testing:
>
> - Docker build - Could not figure out how to get all components to build
> at once
> - build/build.sh on CentOS 6 - all components built successfully
> - Upgraded Traffic Monitor (2.2.0.d7e588 -> 2.2.0.6ea850)
>   - Could pull up UI and it connects to all cache servers
> - Upgraded Traffic Router (2.2.0.d7e588 -> 2.2.0.6ea850)
>   - Could connect to : and crs/stats looked correct
> - Upgraded Traffic Stats (2.2.0.d7e588 -> 2.2.0.6ea850) CentOS 6
> - Upgraded Traffic Ops ORT
>   - mid cache "report" and "badass" mode seem to work correctly
>   - edge cache "report" and "badass" mode seem to work correctly
> - Tested DNS DS and works correctly
>
> On 03/05/2018 01:25 PM, Robert Butts wrote:
>
>> Hello All,
>>
>> I've prepared a release for v2.2.0-RC2
>>
>> The vote is open for at least 72 hours and passes if a majority of at
>> least
>> 3 +1 PPMC votes are cast.
>>
>> [ ] +1 Approve the release
>>
>> [ ] -1 Do not release this package because ...
>>
>> Changes since 2.1.0:
>> https://github.com/apache/incubator-trafficcontrol/compare/
>> RELEASE-2.1.0...RELEASE-2.2.0-RC2
>>
>> This corresponds to git:
>>   Hash: 6ea85056a1a69973be4a74b82d602df29f21f42d
>>   Tag: RELEASE-2.2.0-RC2
>>
>> Which can be verified with the following: git tag -v RELEASE-2.2.0-RC2
>>
>> My code signing key is available here:
>> http://keys.gnupg.net/pks/lookup?search=0xFDD04F7F=vindex
>>
>> Make sure you refresh from a key server to get all relevant signatures.
>>
>> The source .tar.gz file, pgp signature (.asc signed with my key from
>> above), md5 and sha1 checksums are provided here:
>>
>> https://dist.apache.org/repos/dist/dev/incubator/trafficcontrol/2.2.0/RC2
>>
>>
>> Thanks!
>>
>>

Re: [VOTE] Release Apache Traffic Control (incubating) 2.2.0-RC2

[Hank Beatty]

+1

Lab Testing:

- Docker build - Could not figure out how to get all components to build at once
- build/build.sh on CentOS 6 - all components built successfully
- Upgraded Traffic Monitor (2.2.0.d7e588 -> 2.2.0.6ea850)
  - Could pull up UI and it connects to all cache servers
- Upgraded Traffic Router (2.2.0.d7e588 -> 2.2.0.6ea850)
  - Could connect to : and crs/stats looked correct
- Upgraded Traffic Stats (2.2.0.d7e588 -> 2.2.0.6ea850) CentOS 6
- Upgraded Traffic Ops ORT
  - mid cache "report" and "badass" mode seem to work correctly
  - edge cache "report" and "badass" mode seem to work correctly
- Tested DNS DS and works correctly

On 03/05/2018 01:25 PM, Robert Butts wrote:

Hello All,

I've prepared a release for v2.2.0-RC2

The vote is open for at least 72 hours and passes if a majority of at least
3 +1 PPMC votes are cast.

[ ] +1 Approve the release

[ ] -1 Do not release this package because ...

Changes since 2.1.0:
https://github.com/apache/incubator-trafficcontrol/compare/RELEASE-2.1.0...RELEASE-2.2.0-RC2

This corresponds to git:
  Hash: 6ea85056a1a69973be4a74b82d602df29f21f42d
  Tag: RELEASE-2.2.0-RC2

Which can be verified with the following: git tag -v RELEASE-2.2.0-RC2

My code signing key is available here:
http://keys.gnupg.net/pks/lookup?search=0xFDD04F7F=vindex

Make sure you refresh from a key server to get all relevant signatures.

The source .tar.gz file, pgp signature (.asc signed with my key from
above), md5 and sha1 checksums are provided here:

https://dist.apache.org/repos/dist/dev/incubator/trafficcontrol/2.2.0/RC2


Thanks!