Page edited by Chuck Rolke

Comment: Add 'quota queues' file syntax from QPID-4604

Changes (2)

Full Content

v2 ACL file format for brokers

The v2 ACL file format has been designed for implementation and interoperability on all Qpid brokers. It is currently supported in the following brokers:

Broker	Version
C++	M4 onward
Java	M5 onward

This document is updated for release 0.20.

Contents

v2 ACL file format for brokers

Specification

Comments
White Space
Character Set
Case Sensitivity
Line Continuation
Line Length
The all keyword
ACL File Syntax
File Processing Overview

Deny Mode vs. Allow Mode

Rule Processing Details
Rule Matching Details
User Name and Domain Name Symbol Substitution
Routingkey Wildcard

Validation
Example files

Design Documentation

Mapping of ACL traps to action and type

v2 ACL User Guide

Writing Good/Fast ACL
Getting ACL to Log
User Id / domains running with C++ broker
ACL command line options
Quota Settings

Connection quota
Queue quota

Specification

Comments

A line starting with the '#' character is considered a comment and is ignored.
Embedded comments and trailing comments are not allowed.
- The '#' character is commonly found in routing keys and other AMQP literals and occur naturally in ACL rule specifications.
- By disallowing embedded and trailing comments there is less confusion in writing ACL rules that work correctly.

White Space

Blank lines and lines that contain only whitespace (any combination of ' ', '\f', '\n', '\r', '\t', '\v') are ignored.
Additional whitespace between and after tokens is ignored.
Group and acl definition lines must start with "group" or "acl" respectively and with no preceding whitespace.

Character Set

ACL files use are constrained to 7-bit ASCII characters.
Group names may contain only
- a-z
- A-Z
- 0-9
- '-' hyphen
- '_' underscore
Individual user names may contain only
- a-z
- A-Z
- 0-9
- '-' hyphen
- '_' underscore
- '.' period
- '@' ampersand
- '/' slash

Case Sensitivity

All tokens are case sensitive.
"name1" is not equal to "Name1"
"create" is not equal to "CREATE".

Line Continuation

Group name lists may be extended to the following line by terminating the current line with the '\' character. No other ACL file lines may be continued.
Group specification lines may be continued only after the group name or any of the names following the group name. See the example below.
Lines consisting solely of a '\' continuation character are not permitted.
The '\' continuation character is recognized only if it is the last character in the line. Any characters after the '\' are not permitted.

# Examples of extending group lists using a trailing '\' character


group group1 name1 name2 \
             name3 name4 \
             name5

group group2 \
             group1 \
             name6

# The following are illegal:

# '\' must be after group name
group \
      group3 name7 name8

# No empty extension lines
group group4 name9 \
                   \
             name10

Line Length

ACL file lines are limited to 1024 characters.

The all keyword

The keyword "all" is reserved and may be used in ACL rules to match all individuals and groups, all actions, or all objects.
- acl allow all all
- acl deny all all
- acl deny user1 all
- acl allow fred@QPID create all

ACL File Syntax

user = username[/domain[@realm]]
user-list = user1 user2 user3 ...
group-name-list = group1 group2 group3 ...

group <group-name> = [user-list] [group-name-list]

permission = [allow|allow-log|deny|deny-log]
action = ""
object = [virtualhost|queue|exchange|broker|link|route|method]
property = [name|durable|owner|routingkey|passive|autodelete|exclusive|type|alternate|queuename|
            policytype|schemapackage|schemaclass|
            queuemaxsizelowerlimit|queuemaxsizeupperlimit|
            queuemaxcountlowerlimit|queuemaxcountupperlimit|
            filemaxsizelowerlimit|filemaxsizeupperlimit|
            filemaxcountlowerlimit|filemaxcountupperlimit]

acl permission {<group-name>|<user-name>|"all"} {action|"all"} [object|"all" [property=<property-value>]]

quota {connections|queues} N {<group-name>|<user-name>|"all"} [{<group-name>|<user-name>|"all"}]

ACL rules must have the following tokens in order on a single line:
- The string literal "acl"
- The permission
- The name of a single group or of an individual, or the keyword "all"
- The name of an action, or the keyword "all"
- Optionally, a single object name, or the keyword "all"
  - If the object is present, then optionally one or more property name-value pairs in the form 'property=value'.
ACL quotas must have the following tokens in order on a single line:
- The string literal "quota"
- The string literal "connections"
- The number of allowed connections in decimal within the range 0..65530.
- The name of a single group or of an individual, or the keyword "all"
- Optionally, a list of groups, individuals, or the keyword "all"

File Processing Overview

ACL files are processed line by line from the top of the file down.
Group membership must be established before the group name is used in an ACL rule.
Connection quota values are assigned to individuals and to all members of groups in the order specified in the file. Users who received multiple connection quota values use the last value assigned.

Deny Mode vs. Allow Mode

By default ACL files are in 'Deny Mode' and deny all actions by all users. That is, there is an implicit "acl deny all all" rule appended to the ACL rule list. In 'Deny Mode' any action by any actor is denied unless some other rule in the ACL file explicitly allows it.
Administrators may change an ACL file to 'Allow Mode' by ending the ACL file with "acl allow all all". When this rule is present at then end of the ACL file then any action by any actor is allowed unless some other rule in the ACL file explictly denys it.
C++ Brokers prior to release 0.16 had an optimization based on the Allow Mode or Deny Mode.
- In a Deny Mode ACL file all 'deny' or 'deny-log' rules are discarded.
- Similarly in an Allow Mode ACL file all 'allow' or 'allow-log' rules are discarded.
- This characteristic made specification of certain ACL rules difficult and confusing.
C++ Brokers after release 0.16 do not optimize the rules based on Allow Mode or Deny Mode.
- All rules are kept and processed in the order specified my the ACL file.
- Allow rules may be mixed with Deny rules and if a rule is matched then an allow or deny decision is rendered.

# Example of C++ Broker Allow Mode processing behavior

group admins bob@QPID joe@QPID
acl allow bob@QPID create queue
acl deny  admins   create queue
acl allow all      all

# Prior to release 0.16 bob is denied permission to create a queue because the
# 'allow bob@QPID create queue' rule in an Allow Mode file is discarded.
# Since release 0.16 bob is allowed permission to create a queue because no
# rules in the file are discarded and the first allow rule controls the decision.

Rule Processing Details

Rules are interpreted from the top of the file down until a matching rule is obtained. The matching rule then controls the allow or deny decision.

Rule Matching Details

The minimum matching criteria for all ACL rules are:
- An user (name of an individual or of a group)
- An action
- An object
If a rule does not match the minimum criteria then that rule does not control the ACL decision.
ACL rules optionally specify object names and property values:
- If an object is named and/or property values are specified in an ACL rule then all of them must match to cause the rule to match.

# Example of rule matching
#
# Using this ACL file content:

(1)  acl deny bob create exchange name=test durable=true passive=true
(2)  acl deny bob create exchange name=myEx type=direct
(3)  acl allow all all

#
# Lookup 1. id:bob action:create objectType:exchange name=test {durable=false passive=false type=direct alternate=}
#
# ACL Match Processing:
#  1. Rule 1 passes minimum criteria with user bob, action create, and object exchange.
#  2. Rule 1 matches name=test.
#  3. Rule 1 does not match the rule's durable=false with the requested lookup of durable=true.
#  4. Rule 1 does not control the decision and processing continues to Rule 2.
#  5. Rule 2 passes minimum criteria with user bob, action create, and object exchange.
#  6. Rule 2 does not match the rule's name=myEx with the requested lookup of name=test.
#  7. Rule 2 does not control the decision and processing continues to Rule 3.
#  8. Rule 3 matches everything and the decision is 'allow'.
#
# Lookup 2. id:bob action:create objectType:exchange name=myEx {durable=true passive=true type=direct alternate=}
#
# ACL Match Processing:
#  1. Rule 1 passes minimum criteria with user bob, action create, and object exchange.
#  6. Rule 1 does not match the rule's name=test with the requested lookup of name=myEx.
#  4. Rule 1 does not control the decision and processing continues to Rule 2.
#  5. Rule 2 passes minimum criteria with user bob, action create, and object exchange.
#  2. Rule 2 matches name=myEx.
#  3. Rule 2 matches the rule's type=direct with the requested lookup of type=direct.
#  8. Rule 2 matches everything and the decision is 'deny'.
#

C++ Broker new rule matching criteria.
- Certain rule properties are introduced that constrain settings from exceeding ACL minimums or maximums.
- When specified in an ACL allow rule these properties do not affect that rule's match determination. Rather, when the constraint is exceeded then the allow rule which is otherwise matched is converted into a deny rule.
- The new rule properties have no effect in deny rules.
- New rule properties are added in 0.16:
  - queuemaxsizelowerlimit
  - queuemaxsizeupperlimit (alias: maxqueuesize)
  - queuemaxcountlowerlimit
  - queuemaxcountupperlimit (alias: maxqueuecount)
- New rule properties are added in 0.20:
  - filemaxsizelowerlimit
  - filemaxsizeupperlimit
  - filemaxcountlowerlimit
  - filemaxcountupperlimit

This table shows information about the C++ Broker limit properties

User Option	ACL Limit Property Name	Unit
qpid.max_size	queuemaxsizelowerlimit	bytes
qpid.max_size	queuemaxsizeupperlimit	bytes
qpid.max_count	queuemaxcountlowerlimit	messages
qpid.max_count	queuemaxcountupperlimit	messages
qpid.file_size	filemaxsizelowerlimit	pages (64kb/page)
qpid.file_size	filemaxsizeupperlimit	pages (64kb/page)
qpid.file_count	filemaxcountlowerlimit	files
qpid.file_count	filemaxcountupperlimit	files

The following example demonstrates using the size constraints.

# Example of ACL specifying queue size constraints
# Note: for legibility this acl line has been split into multiple lines.

acl allow bob@QPID create queue name=q6 queuemaxsizelowerlimit=500000
                                        queuemaxsizeupperlimit=1000000
                                        queuemaxcountlowerlimit=200
                                        queuemaxcountupperlimit=300

#
# These limits come into play when bob@QPID creates a queue as illustrated here:

    const char* address =

        "message_queue; “
        “ { create: always, “
        “   node: “
        “   { type: queue, “
        “     x-declare: ”
        “     { arguments: “
        “       { qpid.max_count:101,”
        “         qpid.max_size:1000000”
        “       }”
        “     }”
        “   }”
        “ }";
    ...
    Sender sender = session.createSender(address);

#
# When the ACL rule is processed the user, action, object, and object name all match
# and so this allow rule matches for the allow or deny decision. However, the ACL rule
# is further constrained to limit 500000 <= max_size <= 1000000 and
# 200 <= max_count <= 300. Since the queue_option max_count is 101 then the size
# limit is violated and the allow rule is returned with a deny decision.

User Name and Domain Name Symbol Substitution

In the C++ Broker 0.20 a simple set of user name and domain name substitution variable keyword tokens is defined. This provides administrators with an easy way to describe private or shared resources.

Symbol substitution is allowed in the ACL file anywhere that text is supplied for a property value.

In the following table an authenticated user named [email protected] has his substitution keywords expanded.

Keyword	Expansion
${userdomain}	bob_user_QPID_COM
${user}	bob_user
${domain}	QPID_COM

The original user name has the period “.” and ampersand “@” characters translated into underscore “_”. This allows substitution to work when the substitution keyword is used in a routingkey in the Acl file.
The Acl processing matches ${userdomain} before matching either ${user} or ${domain}. Rules that specify the combination ${user}_${domain} will never match.

Example:

Administrators can set up Acl rule files that allow every user to create a private exchange, a private queue, and a private binding between them. In this example the users are also allowed to create private backup exchanges, queues and bindings. This effectively provides limits to user's exchange, queue, and binding creation and guarantees that each user gets exclusive access to these resources.

#
# Create primary queue and exchange:
#
acl allow all create  queue    name=$\{user}-work alternate=$\{user}-work2
acl deny  all create  queue    name=$\{user}-work alternate=*
acl allow all create  queue    name=$\{user}-work
acl allow all create  exchange name=$\{user}-work alternate=$\{user}-work2
acl deny  all create  exchange name=$\{user}-work alternate=*
acl allow all create  exchange name=$\{user}-work
#
# Create backup queue and exchange
#
acl deny  all create  queue    name=$\{user}-work2 alternate=*
acl allow all create  queue    name=$\{user}-work2
acl deny  all create  exchange name=$\{user}-work2 alternate=*
acl allow all create  exchange name=$\{user}-work2
#
# Bind/unbind primary exchange
#
acl allow all bind   exchange name=$\{user}-work routingkey=$\{user} queuename=$\{user}-work
acl allow all unbind exchange name=$\{user}-work routingkey=$\{user} queuename=$\{user}-work
#
# Bind/unbind backup exchange
#
acl allow all bind   exchange name=$\{user}-work2 routingkey=$\{user} queuename=$\{user}-work2
acl allow all unbind exchange name=$\{user}-work2 routingkey=$\{user} queuename=$\{user}-work2
#
# Access primary exchange
#
acl allow all access exchange name=$\{user}-work routingkey=$\{user} queuename=$\{user}-work
#
# Access backup exchange
#
acl allow all access exchange name=$\{user}-work2 routingkey=$\{user} queuename=$\{user}-work2
#
# Publish primary exchange
#
acl allow all publish exchange name=$\{user}-work routingkey=$\{user}
#
# Publish backup exchange
#
acl allow all publish exchange name=$\{user}-work2 routingkey=$\{user}
#
# deny mode
#
acl deny all all

Routingkey Wildcard

In the C++ Broker 0.20 the logic governing the ACL Match have changed for each ACL rule that contains a routingkey property. The routingkey property is matched according to Topic Exchange match logic the broker uses when it distributes messages published to a topic exchange.

Suppose an ACL rule file is:

acl allow-log uHash1@COMPANY publish exchange name=X routingkey=a.#.b
acl deny all all

When user uHash1@COMPANY attempts to publish to exchange X the ACL will return these results:

routingkey in publish to exchange X	result
a.b	allow-log
a.x.b	allow-log
a.x.y.zz.b	allow-log
a.b.	deny
q.x.b	deny

Validation

Note: In the C++ Broker (0.16 and later) the following validation is performed on ACL rule files:

Files are correctly formatted with respect to character sets, white space, continuation, and line length.
Permissions, Actions and Objects are correctly named.
Sufficient tokens are presented for each ACL rule.
Every line in an ACL file is validated and all invalid ACL rules are logged.
Any validation error is fatal and the broker is shut down.

Example files

This example demonstrates usage of groups and other language syntax features.

# Some groups
group admin ted@QPID martin@QPID
group user-consume martin@QPID ted@QPID
group group2 kim@QPID user-consume rob@QPID
group publisher group2 \
                tom@QPID andrew@QPID debbie@QPID

# Connection quota
quota connections 3  all
quota connections 10 publisher user-consume
quota connections 20 admin
quota connections 50 ted@QPID

# Some rules
acl allow carlt@QPID create exchange name=carl.*
acl deny rob@QPID create queue
acl allow guest@QPID bind exchange name=amq.topic routingkey=stocks.ibm.#  owner=self
acl allow user-consume create queue name=tmp.*

acl allow publisher publish all durable=false
acl allow publisher create queue name=RequestQueue
acl allow consumer consume queue durable=true
acl allow fred@QPID create all
acl allow bob@QPID all queue
acl allow admin all
acl deny kim@QPID all
acl allow all consume queue owner=self
acl allow all bind exchange owner=self

# Last (default) rule
acl deny all all

This example shows the required permissions for granting QMF tool access to user username@QPID from node HOSTNAME.

acl allow username@QPID all all name=username

acl allow username@QPID create  queue name=reply-HOSTNAME.* exclusive=true autodelete=true
acl allow username@QPID consume queue name=reply-HOSTNAME.*
acl allow username@QPID bind exchange name=amq.direct routingkey=reply-HOSTNAME.*

acl allow username@QPID create  queue name=topic-HOSTNAME.* exclusive=true autodelete=true
acl allow username@QPID consume queue name=topic-HOSTNAME.*
acl allow username@QPID bind exchange name=qpid.management routingkey=schema.#
acl allow username@QPID bind exchange name=qpid.management routingkey=console.obj.*.*.org.apache.qpid.broker.agent
acl allow username@QPID bind exchange name=qpid.management routingkey=console.event.*.*.org.apache.qpid.broker.agent

acl allow username@QPID create  queue name=qmfc-v2-HOSTNAME.* exclusive=true autodelete=true
acl allow username@QPID consume queue name=qmfc-v2-HOSTNAME.*
acl allow username@QPID bind exchange name=qmf.default.direct routingkey=qmfc-v2-HOSTNAME.*

acl allow username@QPID create  queue name=qmfc-v2-ui-HOSTNAME.* exclusive=true autodelete=true
acl allow username@QPID consume queue name=qmfc-v2-ui-HOSTNAME.*

acl allow username@QPID create  queue name=qmfc-v2-hb-HOSTNAME.* exclusive=true autodelete=true
acl allow username@QPID consume queue name=qmfc-v2-hb-HOSTNAME.*
acl allow username@QPID bind exchange name=qmf.default.topic routingkey=agent.ind.heartbeat.#

acl allow username@QPID publish exchange name=qpid.management    routingkey=broker
acl allow username@QPID publish exchange name=qmf.default.topic  routingkey=console.request.agent_locate
acl allow username@QPID publish exchange name=qmf.default.direct routingkey=broker

acl allow username@QPID access exchange name=qmf.default.direct
acl allow username@QPID access exchange name=qmf.default.topic

acl deny all all

Design Documentation

Mapping of ACL traps to action and type

The C++ broker maps the ACL traps in the follow way for AMQP 0-10.
The Java broker currently only performs ACLs on the AMQP connection not on management functions.
This table also shows which property values are allowed to be specified for each Object-Action pair.

Object	Action	Properties	Trap C++	Trap Java
Exchange	Create	name type alternate passive durable	ExchangeHandlerImpl::declare	ExchangeDeclareHandler
Exchange	Delete	name	ExchangeHandlerImpl::delete	ExchangeDeleteHandler
Exchange	Access	name	ExchangeHandlerImpl::query
Exchange	Bind	name routingkey queuename owner	ExchangeHandlerImpl::bind	QueueBindHandler
Exchange	Unbind	name routingkey	ExchangeHandlerImpl::unbind	ExchangeUnbindHandler
Exchange	Access	name queuename routingkey	ExchangeHandlerImpl::bound
Exchange	Publish	name routingkey	SemanticState::route	BasicPublishMethodHandler
Queue	Access	name	QueueHandlerImpl::query
Queue	Create	name alternate passive durable exclusive autodelete policytype queuemaxsizelowerlimit queuemaxsizeupperlimit queuemaxcountlowerlimit queuemaxcountupperlimit filemaxsizelowerlimit filemaxsizeupperlimit filemaxcountlowerlimit filemaxcountupperlimit	QueueHandlerImpl::declare	QueueDeclareHandler
Queue	Purge	name	QueueHandlerImpl::purge	QueuePurgeHandler
Queue	Purge	name	Management::Queue::purge
Queue	Delete	name	QueueHandlerImpl::delete	QueueDeleteHandler
Queue	Consume	name (possibly add in future?)	MessageHandlerImpl::subscribe	BasicConsumeMethodHandler BasicGetMethodHandler
<Object>	Update		ManagementProperty::set
<Object>	Access		ManagementProperty::read
Link	Create		Management::connect
Route	Create		Management:: -~~createFederationRoute~~-
Route	Delete		Management:: -~~deleteFederationRoute~~-
Virtualhost	Access	name	TBD	ConnectionOpenMethodHandler

Management actions that are not explicitly given a name property it will default the name property to management method name, if the action is 'W' Action will be 'Update', if 'R' Action will be 'Access'.

for example, if the mgnt method 'joinCluster' was not mapped in schema it will be mapped in ACL file as follows

Object	Action	Property
Broker	Update	name=joinCluster

v2 ACL User Guide

Writing Good/Fast ACL

The file gets read top down and rule get passed based on the first match. In the following example the first rule is unnecessary. The second rule is wider than the first rule and the same result would be achieved if the first rule was removed.

acl allow peter@QPID create queue name=tmp <-- unnecessary rule causes extra work.
acl allow peter@QPID create queue
acl deny all all

By default files end with

acl deny all all

the mode of the ACL engine can be swapped to be allow based by putting the following at the end of the file

acl allow all all

Note that 'allow' based file will be a LOT faster for message transfer. This is because the AMQP specification does not allow for creating subscribes on publish, so the ACL is executed on every message transfer. Also, ACL rules using fewer properties on publish will in general be faster.

Getting ACL to Log

In order to get log messages from ACL actions use allow-log and deny-log for example

acl allow-log john@QPID all all
acl deny-log guest@QPID all all

User Id / domains running with C++ broker

The user-id used for ACL is taken from the connection user-id. Thus in order to use ACL the broker authentication has to be setup. i.e. (if --auth no is used in combination with ACL the broker will deny everything)

The user id in the ACL file is of the form <user-id>@<domain> The Domain is configured via the SASL configuration for the broker, and the domain/realm for qpidd is set using --realm and default to 'QPID'.

ACL command line options

To load the ACL module use the --load-module directive on the command line or via the config file

./src/qpidd --load-module src/.libs/acl.so

The ACL plugin provides the following option '--acl-file'. If the ACL file is not supplied then the broker will not enforce ACL. If an ACL file name is supplied, and the file does not exist or is invalid the broker will not start.

ACL Options:
  --acl-file FILE       The policy file to load from, loaded from data dir

Quota Settings

Quota options are integrated with the ACL rule file. The ACL rule file already contains the definition of user and group names and the ACL permissions for each. The ACL rule file additionally uses the same user and group names for the specification of quotas.

Connection quota

Connection quotas work in conjunction with the command line switch '--connection-limit-per-user N' to limit users to some number of concurrent connections.

If the command line switch '--connection-limit-per-user' is absent and there are no 'quota connections' rules in the ACL file then connection limits are not enforced.
If the command line switch '--connection-limit-per-user' is present then it assigns an initial value for the pseudo-user 'all'.
If the ACL file specifies a quota for pseudo user 'all' than that value is applied to all users who are otherwise unnamed in the ACL file.
Connection quotas for users are registered in order as the rule file is processed. A user may be assigned any number of connection quota values but only the final value is retained and enforced.
Connection quotas for groups are applied as connection quotas for each individual user in the group at the time the 'quota connections' line is processed.
Quota values range from 0 to 65530. A value of zero (0) denies connections for that user or group.

Queue quota

Queue quotas work in conjunction with the command line switch '--max-queues-per-user N' to limit users to some number of concurrent queues.

If the command line switch '--max-queues-per-user' is absent and there are no 'quota queues' rules in the ACL file then queue limits are not enforced.
If the command line switch '--max-queues-per-user' is present then it assigns an initial value for the pseudo-user 'all'.
If the ACL file specifies a quota for pseudo user 'all' than that value is applied to all users who are otherwise unnamed in the ACL file.
Queue quotas for users are registered in order as the rule file is processed. A user may be assigned any number of queue quota values but only the final value is retained and enforced.
Queue quotas for groups are applied as queue quotas for each individual user in the group at the time the 'quota queues' line is processed.
Quota values range from 0 to 65530. A value of zero (0) denies queues for that user or group.

Change Notification Preferences

View Online | View Changes | Add Comment

--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

[CONF] Apache Qpid > ACL

ACL