Repository: qpid-dispatch Updated Branches: refs/heads/master 6e559d779 -> 3406c55ed
DISPATCH-311: Rename/reorganize policy schema objects. Add policy writeup in book. Delete stale design notes. Project: http://git-wip-us.apache.org/repos/asf/qpid-dispatch/repo Commit: http://git-wip-us.apache.org/repos/asf/qpid-dispatch/commit/3406c55e Tree: http://git-wip-us.apache.org/repos/asf/qpid-dispatch/tree/3406c55e Diff: http://git-wip-us.apache.org/repos/asf/qpid-dispatch/diff/3406c55e Branch: refs/heads/master Commit: 3406c55eda786b0edc4bb5d790fddec953673532 Parents: 6e559d7 Author: Chuck Rolke <[email protected]> Authored: Fri Jul 22 17:35:24 2016 -0400 Committer: Chuck Rolke <[email protected]> Committed: Fri Jul 22 17:35:24 2016 -0400 ---------------------------------------------------------------------- doc/book/book.adoc | 6 + doc/book/policy.adoc | 368 ++++++++++++++++++++++++++++++ doc/notes/qdr-policy-01.odt | Bin 33139 -> 0 bytes doc/notes/qdr-policy-01.pdf | Bin 212630 -> 0 bytes doc/notes/qdr-policy-diagrams-01.odg | Bin 29492 -> 0 bytes doc/notes/qdr-policy-todo.md | 55 ----- 6 files changed, 374 insertions(+), 55 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/qpid-dispatch/blob/3406c55e/doc/book/book.adoc ---------------------------------------------------------------------- diff --git a/doc/book/book.adoc b/doc/book/book.adoc index 70aba8e..e0618d8 100644 --- a/doc/book/book.adoc +++ b/doc/book/book.adoc @@ -39,6 +39,12 @@ include::auto_links.adoc[] :leveloffset: 0 +:leveloffset: 2 + +include::policy.adoc[] + +:leveloffset: 0 + [[technical-details-and-specifications]] Technical Details and Specifications ------------------------------------ http://git-wip-us.apache.org/repos/asf/qpid-dispatch/blob/3406c55e/doc/book/policy.adoc ---------------------------------------------------------------------- diff --git a/doc/book/policy.adoc b/doc/book/policy.adoc new file mode 100644 index 0000000..bc57fc7 --- /dev/null +++ b/doc/book/policy.adoc @@ -0,0 +1,368 @@ +//// +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License +//// + +[[policy]] += Policy + +The Policy module is an optional authorization mechanism enforcing +user connection restrictions and AMQP resource access control. + +Policy is assigned when a connection is created. The connection +properties *AMQP virtual host*, *authenticated user name*, and *connection +remote host* are passed to the policy engine for a connection +allow/deny decision. If the connection is allowed then the user is +assigned to a group that names a set of AMQP resource limits that are +enforced for the lifetime of the connection. + +[NOTE] +==== +Policy limits are applied only to incoming user network connections. +Policy limits are not applied to interrouter connections nor are they +applied to router connections outbound to waypoints. +==== + +== Definitions + +=== vhost + +A _vhost_ is typically the name of the host to which the client AMQP +connection is directed. For example, suppose a client application opens +connection URL: + +[options="nowrap"] +---- +amqp://bigbroker.example.com:5672/favorite_subject +---- + +The client will signal virtual host name _bigbroker.example.com_ to +the router during AMQP connection startup. Router Policy intercepts +the virtual host _bigbroker.example.com_ and applies a vhost policy +with that name to the connection. + +== Policy Features + +=== Total Connection Limit + +A router may be configured with a total connection limit. This limit +controls the maximum number of simultaneous incoming user connections +that are allowed at any time. It protects the router from file +descriptor resource exhaustion in the face of many incoming client +connections. This limit is specified and enforced independently of +any other Policy settings. + +=== Vhost Policy + +Vhost policy defines users and assigns them to user groups. Each +user group defines the remote hosts from which the members may connect +to the router network, and what resources in the router network the +group members are allowed to access. + +Vhost policy also defines connection count limits to control the +number of users that may be simultaneously connected to the vhost. + +[NOTE] +==== +A vhost user may be assigned to one user group only. +==== + +=== Default Vhost + +A default vhost may be defined. The default vhost policy is used for +connections whose vhost is otherwise not defined in the policy database. + +xref:example2[Example 2] illustrates how the default vhost feature can +be used to apply a single vhost policy set of restrictions to any +number of vhost connections. + +== Policy Schema + +Policy configuration is specified in two schema objects. + +[options="nowrap"] +---- +policy = { + <global settings> +} + +vhost = { + name: vhost name + <connection limits> + groups: { + group-name: { + <user group settings> + } + } +} +---- + +The _policy_ object is a singleton. Multiple _vhost_ objects may be +created as needed. + +=== Global Policy + +[options="header", cols="35,15,50"] +|==== +| attribute | default | description +| maxConnections | 65535 | Global maximum number of concurrent client connections allowed. This limit is always enforced even if no other policy settings have been defined. This limit is applied to all incoming connections regardless of remote host, authenticated user, or targeted vhost. +| enableVhostPolicy | true | Enable vhost policy connection denial, and resource limit enforcement. +| policyDir | "" | Absolute path to a directory that holds vhost definition .json files. All vhost definitions in all .json files in this directory are processed. +| defaultVhost | "$default" | Vhost rule set name to use for connections with a vhost that is otherwise not defined. Default vhost processing may be disabled either by erasing the definition of _defaultVhost_ or by not defining a _vhost_ object named _$default_. +|==== + +=== Vhost Policy + +[options="header", cols="35,15,50"] +|==== +| attribute | default | description +| name | | Vhost name. A unique name is required. +| maxConnections | 65535 | Maximum number of concurrent client connections allowed +| maxConnectionsPerUser | 65535 | Maximum number of concurrent client connections allowed for any single user. +| maxConnectionsPerRemoteHost | 65535 | Maximum number of concurrent client connections allowed for any remote host. +| allowUnknownUser | false | Allow unknown users who are not members of a defined user group. Unknown users are assigned to the '$default' user group and receive '$default' settings. +| groups | | A map where each key is a user group name and the value is a Vhost User Group Settings map. +|==== + +=== Vhost User Group Settings Map + +This object is the data value contained in entries in the policy/groups map. + +[options="header", cols="35,15,50"] +|==== +| Section/Attribute | default | description +| *Group Membership* | | +| users | "" | List of authenticated users in this group +| *Connection Restrictions* | | +| remoteHosts | "" | List of remote hosts from which the users may connect. List values may be host names, numeric IP addresses, numeric IP address ranges, or the wildcard '*'. An empty list denies all access. +| *AMQP Connection Open Limits* | | +| maxFrameSize | 2^32-1 | Largest frame that may be sent on this connection. (AMQP Open, max-frame-size) +| maxSessions | 65535 | Maximum number of sessions that may be created on this connection. (AMQP Open, channel-max) +| *AMQP Session Begin Limits* | | +| maxSessionWindow | 2^32-1 |Incoming capacity for new sessions. (AMQP Begin, incoming-window) +| *AMQP Link Attach* | | +| maxMessageSize | 0 | Largest message size supported by links created on this connection. If this field is zero there is no maximum size imposed by the link endpoint. (AMQP Attach, max-message-size) +| maxSenders | 2^32-1 | Maximum number of sending links that may be created on this connection. +| maxReceivers | 2^32-1 | Maximum number of receiving links that may be created on this connection. +| allowDynamicSource | false | This connection is allowed to create receiving links using the Dynamic Link Source feature. +| allowAnonymousSender | false | This connection is allowed to create sending links using the Anonymous Sender feature. +| sources | "" | List of Source addresses allowed when creating receiving links. This list may be expressed as a CSV string or as a list of strings. An empty list denies all access. +| targets | "" | List of Target addresses allowed when creating sending links. This list may be expressed as a CSV string or as a list of strings. An empty list denies all access. +|==== + +== Policy Wildcard and User Name Substitution + +Policy provides several conventions to make writing rules easier. + +=== Remote Host Wildcard + +Remote host rules may consist of a single asterisk character to +specify all hosts. + +[options="nowrap"] +---- + remoteHosts: * +---- + +The asterisk must stand alone and cannot be appended to a host name +or to an IP address fragment. + +=== User Name Wildcard + +User names in group name lists may be specified by a name prefix +terminated with a single asterisk character. + +[options="nowrap"] +---- + users: customer*, admin, employee* +---- + +=== AMQP Source and Target Wildcard and Name Substitution + +The rule definitions for `sources` and `targets` may include the token +'*{user}*' or a trailing asterisk. + +The _{user}_ token is replaced with the authenticated user name for +the connection. Using this token, an administrator may allow access to +some resources specific to each user without having to name each user +individually. This token is substituted once for the leftmost +occurrence in the link name. + +The asterisk is recognized only if it is the last character in the +link name. + +[options="nowrap"] +---- + sources: tmp_{user}, temp*, {user}-home-* +---- + +== Composing Policies + +This section shows policy examples designed to illustrate some common use cases. + +=== Example 1. User Policy Disabled + +Policy is disabled when no policy configuation objects are defined. +Any number of connections are allowed and all users have +access to all AMQP resources in the network. + +[[example2]] +=== Example 2. All Users Have Simple Connection Limits + +This example shows how to keep users from overwhelming the router with +connections. Any user can create up to ten connections and the router +will limit the aggregated user connection count to 100 connections +total. No other restrictions apply. + +This example also shows how to use a default vhost policy effectively. +Only one vhost policy is defined and all user connections regardless +of the requested vhost use that policy. + +[options="nowrap"] +---- +policy { + maxConnections: 100 <1> +} + +vhost { + name: $default <2> + maxConnectionsPerUser: 10 <3> + groups: { + all: { + users: * <4> + remoteHosts: * <5> + sources: * <6> + targets: * <6> + } + } +} +---- + +<1> The global maxConnections limit of 100 is enforced. +<2> No normal vhost names are defined; user is assigned to default vhost '$default'. +<3> The vhost maxConnectionsPerUser limit of 10 is enforced. +<4> User name is found in user group 'all' +<5> The user is allowed to connect from any remote host. The connection is approved and the user gets user group 'all' settings. +<6> The user is allowed to connect to any source or target in the AMQP network. Router system-wide values are used for the other AMQP settings that are unspecified in the vhost rules. + +=== Example 3. Admins Must Connect From Localhost + +This example shows how an admin group may be created and restricted +to accessing a vhost only from localhost. The admin users are allowed +to connect to any AMQP resources while normal users are restricted. + +In this example a user connects to vhost 'example.com'. + +[options="nowrap"] +---- +vhost { + name: example.com <1> + allowUnknownUser: true <3> + groups: { + admin: { + users: alice, bob <2> + remoteHosts: 127.0.0.1, ::1 <4> + sources: * <5> + targets: * <5> + }, + $default: { + remoteHosts: * <6> + sources: news*, sports*, chat* <7> + targets: chat* <7> + } + } +} +---- + +<1> A connection to vhost 'example.com' locates this vhost rule set. +<2> If one of users _alice_ or _bob_ is connecting then she or he is assigned to the 'admin' user group. +<3> Any other user is not defined by a user group. However, since the _allowUnknownUser_ setting is true then this user is assigned to the '$default' user group. +<4> Users in the 'admin' user group must connect from localhost. Connections for an 'admin' user from other hosts on the network are denied. +<5> Users in the 'admin' user group are allowed to access any resource offered by the vhost service. +<6> Other users are allowed to connect from any host. +<7> Other users have source and target name lists that restrict the resources they are allowed to access. + +=== Example 4. Limiting Possible Memory Consumption + +Policy provides a mechanism to control how much system buffer memory a +user connection can potentially consume. The formula for computing +buffer memory consumption for each session is + +[optons="nowrap"] +---- + potential buffer usage = maxFrameSize * maxSessionWindow +---- + +By adjusting _maxFrameSize_, _maxSessions_, and _maxSessionWindow_ an +administrator can prevent a user from consuming too much memory by +buffering messages in flight. + +[NOTE] +==== +The settings passed into the AMQP protocol connection and session +negotiations. Normal AMQP session flow control limits buffer +consumption in due course with no processing cycles required by the +router. +==== + +In this example normal users, the traders, are given smaller buffer +allocations while high-capacity, automated data feeds are given a +higher buffer allocation. This example skips the details of settings +unrelated to buffer allocation. + +[options="nowrap"] +---- +vhost { + name: traders.com <1> + groups: { + traders: { + users: trader-* <2> + maxFrameSize: 10000 <3> + maxSessionWindow: 500 <3> + maxSessions: 1 <4> + ... + }, + feeds: { + users: nyse-feed, nasdaq-feed <5> + maxFrameSize: 60000 <6> + maxSessionWindow: 20000 <6> + maxSessions: 3 <7> + ... + } + } +} +---- + +<1> These rules are for vhost traders.com. +<2> The 'traders' group includes any user whose name begins with _trader-_. +<3> _maxFrameSize_ and _maxSessionWindow_ allow for at most 5,000,000 bytes of data to be in flight on each session. +<4> Only one session per connection is allowed. +<5> In the 'feeds' group two users are defined. +<6> _maxFrameSize_ and _maxSessionWindow_ allow for at most 1,200,000,000 bytes of data to be in flight on each session. +<7> Up to three sessions per connection are allowed. + +//// + +- Should we used signed ints for limits to allow for magic values? + +- In example 3, alice and bob cannot connect from an outside server and + get the $default settings. + +- Limits are enforced per router, so a vhost policy of maxConnections + 10 across 10 routers will allow 100 connections to that vhost. + +//// http://git-wip-us.apache.org/repos/asf/qpid-dispatch/blob/3406c55e/doc/notes/qdr-policy-01.odt ---------------------------------------------------------------------- diff --git a/doc/notes/qdr-policy-01.odt b/doc/notes/qdr-policy-01.odt deleted file mode 100644 index 24b369e..0000000 Binary files a/doc/notes/qdr-policy-01.odt and /dev/null differ http://git-wip-us.apache.org/repos/asf/qpid-dispatch/blob/3406c55e/doc/notes/qdr-policy-01.pdf ---------------------------------------------------------------------- diff --git a/doc/notes/qdr-policy-01.pdf b/doc/notes/qdr-policy-01.pdf deleted file mode 100644 index bec4e37..0000000 Binary files a/doc/notes/qdr-policy-01.pdf and /dev/null differ http://git-wip-us.apache.org/repos/asf/qpid-dispatch/blob/3406c55e/doc/notes/qdr-policy-diagrams-01.odg ---------------------------------------------------------------------- diff --git a/doc/notes/qdr-policy-diagrams-01.odg b/doc/notes/qdr-policy-diagrams-01.odg deleted file mode 100644 index a259688..0000000 Binary files a/doc/notes/qdr-policy-diagrams-01.odg and /dev/null differ http://git-wip-us.apache.org/repos/asf/qpid-dispatch/blob/3406c55e/doc/notes/qdr-policy-todo.md ---------------------------------------------------------------------- diff --git a/doc/notes/qdr-policy-todo.md b/doc/notes/qdr-policy-todo.md deleted file mode 100644 index de35609..0000000 --- a/doc/notes/qdr-policy-todo.md +++ /dev/null @@ -1,55 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. - -Policy TODO List -================ - -# Ruleset schema - -- Should relesets be able to share definitions? The rulesets are self contained. Rulesets uniquely define policy for one application. Redefinition of a ruleset will have no effect on other applications. The implication is that if one ruleset has a large definition of users, user groups, or policy settings then these definitions can not be shared with other rulesets and *they must be repeated*. - -- Are the policy settings adequate? Do they provide enough controls over AMQP? - - - The policy can allow or deny access to source/target **$management**. But once that access is granted the user can do anything to any object on any router. Finer control over management is desirable. - - - Are user connection counts specified at the proper level? They are specified per application but the counts are applied equally to users in all user groups. - -- Are white list rules (only allow rules) enough? Having only white list rules makes rulesets (1) easier to comprehend and (2) easier to generate. The code to process them is also (3) easier to write and (4) easier to test. Mixed allow and deny rules provide a richer rule specification but with a cost impacting each of the benefit points. - -- Policy adds map schema objects. The schema processor verifies that the objects are maps but does not look into the map to see if it makes any sense for policy. Policy code thoroughly validates the maps but it would be better if the maps were specified in the management schema and verified there. - - - Schema map objects forced modifying the config processor to allow .json files in addition to the existing .conf files. - -# Policy service - -- Current policy is local only. Certainly policy rulesets are manageable entities and may be loaded through normal management channels. There is an opportunity to develop a policy service where routers could open a channel to **$policy** for user lookup and policy settings. Then each router would not need to maintain local policy in its current form. - -# Implementation - -- Effecting denials of application connections, sessions, and links are all handled in the same way. Take an Open for instance. If an Open is denied then QDR returns an Open to the client followed immediately with a Close that has a condition and a description. In all cases the condition is amqp:resource-limit-exceeded. An improvement would be to use amqp:unauthorized-access for denials that are hard policy denials and use resource-limit-exceeded for connections denied by connection counts that may change later. - -- Locking. Needs review. - -- Memory pool for policy settings. Uses alloc/free for each connection for small setting structs. Low priority. - -- Add default 'default settings'. In the absence of policy settings, or in the presence of policies without some settings, dispatch router needs to insert values into AMQP fields. The current implementation hard codes these but they should all be configurable. - -# Usual suspects - -- Improve documentation. - -- Improve self tests. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
