[GitHub] qpid-dispatch issue #420: DISPATCH-1177 - Added new option -e to qdstat. Thi...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/420 The problem was router INTC was running master branch code and did not have this PR's changes. With both routers running DISPATCH-1177-1@0e4cd the command works ok. This PR could be improved if there was a way to give a meaningful error message instead of KeyError. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #420: DISPATCH-1177 - Added new option -e to qdstat. Thi...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/420 If I have a network of interior routers, each with some edge routers, I can qdstat -e each OK. But if I add -r some-other-router and the other router is on another host them qdstat fails with **KeyError: 'active'.** > qdstat -e -b amqp://$INTB_normal -r INTB Connections id hostcontainer role dir security authentication tenant active === 2 127.0.0.1:58780 EB2edge in no-security anonymous-user no 72 192.168.1.16:33324 EB1edge in no-security anonymous-user no > qdstat -e -b amqp://$INTB_normal -r INTC KeyError: 'active' --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #421: Import crolke's adverbl project
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/421 Import crolke's adverbl project Scraper is a data scraping program. It reads qpid-dispatch router log files, categorizes and sorts the data, and produces an HTML summary. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch adverbl-log-scraper Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/421.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #421 commit e3f49322eaad43167bd44574fec0527ce6a8f920 Author: Chuck Rolke Date: 2018-11-13T20:53:37Z Import crolke's adverbl project --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #384: DISPATCH-1096 - priority sheaf cleanup bug
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/384 This patch clears up the too-many-links for sheaf error and improves my router network stability. FOr this I approve the patch. But in tests before and after this patch my throughput went from 2600 msg/S to 200-300 msg/S. The performance drop needs to be addressed as a separate issue. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #383: DISPATCH-1129 - Moved call to forwarding multicast...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/383 +1 Ship It Looks good and works. Failing tests could never get beyond a few hundred messages. With this patch I'm past 100 million messages. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-proton pull request #158: PROTON-1937: Use value.type() to check for va...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-proton/pull/158 PROTON-1937: Use value.type() to check for value presence or absence This works for me on Fedroa 27 and 28 You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-proton PROTON-1937 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-proton/pull/158.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #158 commit ef657e67443e067d9fc74920960e6ce99c65f07c Author: Chuck Rolke Date: 2018-09-21T20:39:33Z PROTON-1937: Use value.type() to check for value presence or absence --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #355: DISPATCH-1098: Adjust name of unit2 executa...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/355 DISPATCH-1098: Adjust name of unit2 executable for python3 CMake chooses between 'python' and 'python3'. Make a similar choice for 'unit2' and 'unit2-3'. This lets both python2 and python3 versions to be installed on a system and chooses between them based on the result of PYTHON_VERSING_MAJOR. Tested on Fedora only. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-1098 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/355.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #355 commit fdefed692a8c5a520aebcb0f16551ac0ee92858e Author: Chuck Rolke Date: 2018-08-09T19:00:34Z DISPATCH-1098: Adjust name of unit2 executable for python3 --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #342: DISPATCH-1067 - Improvements and updates to...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/342#discussion_r202471950
--- Diff: docs/books/user-guide/configuration-security.adoc ---
@@ -417,290 +417,367 @@ For more information about these attributes, see
xref:adding-sasl-authentication
== Authorizing Access to Messaging Resources
-You can restrict the number of user connections, and control access to
AMQP messaging resources by configuring _policies_.
+You can configure _policies_ to secure messaging resources in your
messaging environment. Policies ensure that only authorized users can access
messaging endpoints through the router network, and that the resources on those
endpoints are used in an authorized way.
-=== Types of Policies
-
-You can configure two different types of policies: _global policies_ and
_vhost policies_.
+{RouterName} provides the following types of policies:
Global policies::
-Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all vhost policies), and
defines how the router should use vhost policies.
+Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all messaging endpoints), and
defines how the router should use vhost policies.
Vhost policies::
-Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or _vhost_). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
-+
-[NOTE]
-
-A vhost is typically the name of the host to which the client connection
is directed. For example, if a client application opens a connection to the
`amqp://mybroker.example.com:5672/queue01` URL, the vhost would be
`mybroker.example.com`.
-
+Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or vhost). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
The resource limits defined in global and vhost policies are applied to
user connections only. The limits do not affect inter-router connections or
router connections that are outbound to waypoints.
-=== How {RouterName} Applies Policies
+=== How {RouterName} Enforces Connection and Resource Limits
-{RouterName} uses both global and vhost policies to determine whether to
permit a connection, and if it is permitted, to apply the appropriate resource
limits.
+{RouterName} uses policies to determine whether to permit a connection,
and if it is permitted, to apply the appropriate resource limits.
When a client creates a connection to the router, the router first
determines whether to allow or deny the connection. This decision is based on
the following criteria:
-* Whether the connection will exceed the router's global connection limit
(defined in the global policy)
-* Whether the connection will exceed the vhost's connection limits
(defined in the vhost policy that matches the host to which the connection is
directed)
+* Whether the connection will exceed the routerâs global connection
limit (defined in the global policy)
-If the connection is allowed, the router assigns the user (the
authenticated user name from the connection) to a user group, and enforces the
user group's resource limits for the lifetime of the connection.
+* Whether the connection will exceed the vhostâs connection limits
(defined in the vhost policy that matches the host to which the connection is
directed)
-=== Configuring Global Policies
+If the connection is allowed, the router assigns the user (the
authenticated user name from the connection) to a user group, and enforces the
user groupâs resource limits for the lifetime of the connection.
-You can set the incoming connection limit for the router and define how it
should use vhost policies by configuring a global policy.
+=== Setting Global Connection Limits
+
+You can set the incoming connection limit for the router. This limit
defines the total number of concurrent client connections that can be open for
this router.
.Procedure
-* In the router configuration file, add a `policy` section.
+* In the router configuration file, add a `policy` section and set the
`maxConnections`.
+
--
[options="nowrap",subs="+quotes"]
-policy = {
-maxConnections: 1 // <1>
-enableVhostPolicy: true // <2>
-policyDir: /etc/qpid-dispatch/policies/ // <3>
-defaultVhost: $default // <4>
+policy {
+maxConnections: 1
}
-<1> The maximum number of concurrent client connections allow
[GitHub] qpid-dispatch pull request #342: DISPATCH-1067 - Improvements and updates to...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/342#discussion_r202473609
--- Diff: docs/books/user-guide/configuration-security.adoc ---
@@ -417,290 +417,367 @@ For more information about these attributes, see
xref:adding-sasl-authentication
== Authorizing Access to Messaging Resources
-You can restrict the number of user connections, and control access to
AMQP messaging resources by configuring _policies_.
+You can configure _policies_ to secure messaging resources in your
messaging environment. Policies ensure that only authorized users can access
messaging endpoints through the router network, and that the resources on those
endpoints are used in an authorized way.
-=== Types of Policies
-
-You can configure two different types of policies: _global policies_ and
_vhost policies_.
+{RouterName} provides the following types of policies:
Global policies::
-Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all vhost policies), and
defines how the router should use vhost policies.
+Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all messaging endpoints), and
defines how the router should use vhost policies.
Vhost policies::
-Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or _vhost_). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
-+
-[NOTE]
-
-A vhost is typically the name of the host to which the client connection
is directed. For example, if a client application opens a connection to the
`amqp://mybroker.example.com:5672/queue01` URL, the vhost would be
`mybroker.example.com`.
-
+Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or vhost). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
The resource limits defined in global and vhost policies are applied to
user connections only. The limits do not affect inter-router connections or
router connections that are outbound to waypoints.
-=== How {RouterName} Applies Policies
+=== How {RouterName} Enforces Connection and Resource Limits
-{RouterName} uses both global and vhost policies to determine whether to
permit a connection, and if it is permitted, to apply the appropriate resource
limits.
+{RouterName} uses policies to determine whether to permit a connection,
and if it is permitted, to apply the appropriate resource limits.
When a client creates a connection to the router, the router first
determines whether to allow or deny the connection. This decision is based on
the following criteria:
-* Whether the connection will exceed the router's global connection limit
(defined in the global policy)
-* Whether the connection will exceed the vhost's connection limits
(defined in the vhost policy that matches the host to which the connection is
directed)
+* Whether the connection will exceed the routerâs global connection
limit (defined in the global policy)
-If the connection is allowed, the router assigns the user (the
authenticated user name from the connection) to a user group, and enforces the
user group's resource limits for the lifetime of the connection.
+* Whether the connection will exceed the vhostâs connection limits
(defined in the vhost policy that matches the host to which the connection is
directed)
-=== Configuring Global Policies
+If the connection is allowed, the router assigns the user (the
authenticated user name from the connection) to a user group, and enforces the
user groupâs resource limits for the lifetime of the connection.
-You can set the incoming connection limit for the router and define how it
should use vhost policies by configuring a global policy.
+=== Setting Global Connection Limits
+
+You can set the incoming connection limit for the router. This limit
defines the total number of concurrent client connections that can be open for
this router.
.Procedure
-* In the router configuration file, add a `policy` section.
+* In the router configuration file, add a `policy` section and set the
`maxConnections`.
+
--
[options="nowrap",subs="+quotes"]
-policy = {
-maxConnections: 1 // <1>
-enableVhostPolicy: true // <2>
-policyDir: /etc/qpid-dispatch/policies/ // <3>
-defaultVhost: $default // <4>
+policy {
+maxConnections: 1
}
-<1> The maximum number of concurrent client connections allow
[GitHub] qpid-dispatch pull request #342: DISPATCH-1067 - Improvements and updates to...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/342#discussion_r202473464
--- Diff: docs/books/user-guide/configuration-security.adoc ---
@@ -417,290 +417,367 @@ For more information about these attributes, see
xref:adding-sasl-authentication
== Authorizing Access to Messaging Resources
-You can restrict the number of user connections, and control access to
AMQP messaging resources by configuring _policies_.
+You can configure _policies_ to secure messaging resources in your
messaging environment. Policies ensure that only authorized users can access
messaging endpoints through the router network, and that the resources on those
endpoints are used in an authorized way.
-=== Types of Policies
-
-You can configure two different types of policies: _global policies_ and
_vhost policies_.
+{RouterName} provides the following types of policies:
Global policies::
-Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all vhost policies), and
defines how the router should use vhost policies.
+Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all messaging endpoints), and
defines how the router should use vhost policies.
Vhost policies::
-Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or _vhost_). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
-+
-[NOTE]
-
-A vhost is typically the name of the host to which the client connection
is directed. For example, if a client application opens a connection to the
`amqp://mybroker.example.com:5672/queue01` URL, the vhost would be
`mybroker.example.com`.
-
+Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or vhost). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
The resource limits defined in global and vhost policies are applied to
user connections only. The limits do not affect inter-router connections or
router connections that are outbound to waypoints.
-=== How {RouterName} Applies Policies
+=== How {RouterName} Enforces Connection and Resource Limits
-{RouterName} uses both global and vhost policies to determine whether to
permit a connection, and if it is permitted, to apply the appropriate resource
limits.
+{RouterName} uses policies to determine whether to permit a connection,
and if it is permitted, to apply the appropriate resource limits.
When a client creates a connection to the router, the router first
determines whether to allow or deny the connection. This decision is based on
the following criteria:
-* Whether the connection will exceed the router's global connection limit
(defined in the global policy)
-* Whether the connection will exceed the vhost's connection limits
(defined in the vhost policy that matches the host to which the connection is
directed)
+* Whether the connection will exceed the routerâs global connection
limit (defined in the global policy)
-If the connection is allowed, the router assigns the user (the
authenticated user name from the connection) to a user group, and enforces the
user group's resource limits for the lifetime of the connection.
+* Whether the connection will exceed the vhostâs connection limits
(defined in the vhost policy that matches the host to which the connection is
directed)
-=== Configuring Global Policies
+If the connection is allowed, the router assigns the user (the
authenticated user name from the connection) to a user group, and enforces the
user groupâs resource limits for the lifetime of the connection.
-You can set the incoming connection limit for the router and define how it
should use vhost policies by configuring a global policy.
+=== Setting Global Connection Limits
+
+You can set the incoming connection limit for the router. This limit
defines the total number of concurrent client connections that can be open for
this router.
.Procedure
-* In the router configuration file, add a `policy` section.
+* In the router configuration file, add a `policy` section and set the
`maxConnections`.
+
--
[options="nowrap",subs="+quotes"]
-policy = {
-maxConnections: 1 // <1>
-enableVhostPolicy: true // <2>
-policyDir: /etc/qpid-dispatch/policies/ // <3>
-defaultVhost: $default // <4>
+policy {
+maxConnections: 1
}
-<1> The maximum number of concurrent client connections allow
[GitHub] qpid-dispatch pull request #340: Dispatch-1055: Router fails to propagage cr...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/340 Dispatch-1055: Router fails to propagage credit on link router after drain cycle In this PR the "fix" is all in transfer.c. * The router was not accounting for transfers that never arrive when drain consumes the credit without sending messages. The fix is to reset the internal credit counts when a drain cycle ends. * The router calculates if drain changed more correctly. The commit also adds a test that works when not in link route and fails in a link route. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-1055-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/340.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #340 commit baa4d92a9cf6e3ce92199df60a009119b834fd96 Author: Chuck Rolke Date: 2018-06-29T21:43:24Z DISPATCH-1055: Add a test that demonstrates the credit error The test is run first without a link route to observe that it works. Then it is run with the link route and hangs. commit a9b37f030785d1bdd4d9900b5f0be41ded1b2aa9 Author: Chuck Rolke Date: 2018-07-09T20:23:08Z DISPATCH-1055: Fix drain propagated over link route Reset incremental credit given to core at end of drain cycle. Fix logic computing drain_changed flag. commit 52df18eff7f6429ab6fd99ec777136153423afe1 Author: Chuck Rolke Date: 2018-07-09T21:05:51Z fix drain_changed logic --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #334: Dispatch 1046 - fix policy handling in python3 env...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/334 This patch fixes the c_char_p in py3 and improves interfaces with some const additions. The patch also repairs some incorrect handling of policy's parse tree. It looks like there were resource leaks when the trees were deleted. I don't think that was just a py3 issue. This PR looks OK to me. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #324: DISPATCH-976 - Fixed issue with policy validation ...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/324 Looking at it again I think all three of these need_check_* booleans are suspect. Don't delete just the one, get rid of them all. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #311: Dispatch 1011- Policy username substition f...
GitHub user ChugR opened a pull request:
https://github.com/apache/qpid-dispatch/pull/311
Dispatch 1011- Policy username substition failures; Coverity issues
## Problem statement
* The defect is described in
[DISPATCH-1011](https://issues.apache.org/jira/browse/DISPATCH-1011)
* Coverity pointed out the possibility of an unterminated string.
## Resolution
Both issues are addressed with this PR.
* Username substitutions are processed at creation time to divide the
policy rule into a prefix before the username and a suffix after the user name.
Now there is no ambiguity about where the username substitution goes during
policy application to proposed link source or target addresses.
* Username substitution tokens in match patterns are limited in form and
placement to eliminate ambiguity in conjunction with address match wildcards.
* The code Coverity observed is replaced wholesale.
## Restrictions
This PR adds some restrictions to the use of username substitutions in
source and target address patterns.
* The username substitution token must stand alone as a delimited search
token. It may not be concatenated with fixed text strings. Example: token forms
_abc-${user}_ and _${user}-xyz_ are not allowed.
* The username substitution token may only be specified as the first or the
last token in a rule clause. It may not be embedded somewhere in the middle of
a rule. Example: match patterns _#.${user}.*_ and
_fa.#.la.lala.${user}.la.#.la.lalala_ are not allowed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-1011
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/qpid-dispatch/pull/311.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #311
commit 864ed3af3d0c69a1f62444a7cd056db137841a77
Author: Chuck Rolke
Date: 2018-05-31T17:35:49Z
DISPATCH-1011: Improve user name substitution token logic and code
Remove code flagged by Coverity.
Add scheme that specifies precisely where user name substitution goes.
commit 74251cd14a5e5ee8163dd2796f6516afca76ed96
Author: Chuck Rolke
Date: 2018-05-31T18:02:18Z
DISPATCH-1011: Describe user name substitution changes in doc
commit 7c9798135e728143377be54ce18313bf2c60894c
Author: Chuck Rolke
Date: 2018-05-31T19:52:39Z
DISPATCH-1011: Parse tree exports separators. Improve self tests.
Test sourcePattern suffixes found latent paste error in code.
More through test cases to hit more conditional code paths.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #309: DISPATCH-1013 - Enable vhost policies to be define...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/309 This is great. Ship it! An addition to consider for the future: policy supports a _policyDir_ wherein all .json files are read for policy. This could be expanded to include .conf files as well. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #305: Dispatch 965 - Port to Python 3.x
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/305 I ran this branch OK on two systems: * Fedora 27, Python 2.7 * Fedora 28, Python 3.6 The F27 system is my development laptop. It has python 2.6, 2.7, 3.3, 3.4, 3.4m, 3.5 3.5m, 3.6, and 3.6m. I believe that my Dispatch build runs python2.7 but to be honest I can't be sure given all the subprocesses that get launched in dispatch tests. The F28 system was carefully groomed to have only python 3.6 and 3.6m. On that system I'm confident that the Dispatch tests ran python 3 all the way. In particular the dispatch run-time picked up embedded python3. I did not run tox on this system. Tox wants to bring along all kinds of other versions of python and I wanted as pure of a python3-only system as I could get. Great work. Ship it. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #304: Dispatch 990 Use patterns for policy vhost ...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/304 Dispatch 990 Use patterns for policy vhost hostnames This adds all the code (schema, management, run time) and supporting self tests and old style doc. This does not add anything to doc/new-book. I will coordinate that with @bhardesty You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-990 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/304.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #304 commit 32b06d23e29a692f2821f39d02be0f9d092b1648 Author: Chuck Rolke <crolke@...> Date: 2018-05-08T14:37:35Z DISPATCH-990: vhost hostname patterns management interface commit 862b0214991a8d4a6145b5a50980d7613941b2e5 Author: Chuck Rolke <crolke@...> Date: 2018-05-08T17:46:45Z DISPATCH-990: reduce line lengths commit 10ad8cf370412ec424f92e1a3db0aee031f4afc4 Author: Chuck Rolke <crolke@...> Date: 2018-05-10T14:17:21Z DISPATCH-990: Reorder config steps to init policy Policy manager needs some setup before vhosts are processed. commit e1ae8d3022d680775a5f441ea677b1498453ee15 Author: Chuck Rolke <crolke@...> Date: 2018-05-10T19:41:21Z DISPATCH-990: Add c code to support tree hostname lookups commit f90c2a805d663fb2ea15a0fec7c70c16ce0aa1aa Author: Chuck Rolke <crolke@...> Date: 2018-05-11T13:22:24Z DISPATCH-990: Add/remove policy hostname tree entries Tie hostname lookup tree to life cycle of policy rulesets. commit f17631cbe7a695fb4492a4437289d82a1a89a1c0 Author: Chuck Rolke <crolke@...> Date: 2018-05-11T15:10:49Z DISPATCH-990: integrate the pattern match with vhost lookups commit 9c0a09d793795acbc165df37a45f8fa86329da0f Author: Chuck Rolke <crolke@...> Date: 2018-05-14T16:31:13Z DISPATCH-990: Reject add of vhost that has pattern conflict. Add self test showing how conflicts are managed. commit 2b0f3d066428f5c13eed83848f017c39ca56f565 Author: Chuck Rolke <crolke@...> Date: 2018-05-14T16:57:20Z DISPATCH-990: Fix locks so hostname adds are atomic commit 82cef3abff91857871ba7e9ca88177a39db0f7a0 Author: Chuck Rolke <crolke@...> Date: 2018-05-14T18:11:30Z DISPATCH-990: Add hostname translation to settings lookup commit 415b4568ecac113f3ea4212507407dba03f7cdd1 Author: Chuck Rolke <crolke@...> Date: 2018-05-14T19:56:58Z DISPATCH-990: Add hostname pattern self test Remove stray variables from log message. commit 8712cabdf8f37dddc7f8d1ffc4792fbd177f050d Author: Chuck Rolke <crolke@...> Date: 2018-05-14T20:14:03Z DISPATCH-990: add policy dir to support self test commit 63a56d6052e747a0043df621a67805989de5d255 Author: Chuck Rolke <crolke@...> Date: 2018-05-15T14:43:44Z DISPATCH-990: Document name pattern match feature in old book. Rename and touch up configuration settings to make docs read better. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #265: DISPATCH-937 - Introduce the validate flag at the ...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/265 +1 Ship it --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #271: DISPATCH-945: Prevent crash at shutdown whe...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/271 DISPATCH-945: Prevent crash at shutdown when websocket client connected Execute qd_http_server_free before the server is shut down so that lws vhost and context destruction activity is handled properly. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-945-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/271.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #271 commit 860f9abac764a3838661b0c23fa2438d11d7fdab Author: Chuck Rolke <crolke@...> Date: 2018-03-21T17:48:10Z DISPATCH-945: Prevent crash at shutdown when websocket client connected Execute qd_http_server_free before the server is shut down so that lws vhost and context destruction activity is handled properly. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #255: DISPATCH-333: Create new router policies do...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/255#discussion_r171883800
--- Diff: doc/new-book/configuration-security.adoc ---
@@ -412,3 +414,356 @@ listener {
For more information about these attributes, see
xref:adding_sasl_authentication_to_incoming_connection[].
--
+
+== Authorizing Access to Messaging Resources
+
+You can restrict the number of user connections, and control access to
AMQP messaging resources by configuring _policies_.
+
+=== Types of Policies
+
+You can configure two different types of policies: _global policies_ and
_vhost policies_.
+
+Global policies::
+Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all vhost policies), and
defines how the router should use vhost policies.
+
+Vhost policies::
+Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or _vhost_). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
++
+[NOTE]
+
+A vhost is typically the name of the host to which the client connection
is directed. For example, if a client application opens a connection to the
`amqp://mybroker.example.com:5672/queue01` URL, the vhost would be
`mybroker.example.com`.
+
+
+The resource limits defined in global and vhost policies are applied to
user connections only. The limits do not affect inter-router connections or
router connections that are outbound to waypoints.
+
+=== How {RouterName} Applies Policies
+
+When a client connects to a router, the router determines whether to
permit the connection based on the global and vhost policies, and the following
properties of the connection:
+
+* The host to which the connection is directed (the vhost)
+* The connection's authenticated user name
+* The host from which the client is connecting (the remote host)
+
+If the connection is permitted, then the router applies a vhost policy
that matches the vhost to which the connection is directed. The vhost policy
limits are enforced for the lifetime of the connection.
+
--- End diff --
This description is correct but it glosses over some of the structure
within a vhost policy. In a vhost policy maxConnections,
maxConnectionsPerUser, maxConnectionsPerHost, and allowUnknownUser are common
for all users. Then based on the user name the vhost policy assigns the
remaining policy settings (vhostUserGroupSettings). Users who connect to a
given vhost may receive different settings based on what user group the user is
assigned.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #255: DISPATCH-333: Create new router policies do...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/255#discussion_r171881914
--- Diff: doc/new-book/configuration-security.adoc ---
@@ -412,3 +414,356 @@ listener {
For more information about these attributes, see
xref:adding_sasl_authentication_to_incoming_connection[].
--
+
+== Authorizing Access to Messaging Resources
+
+You can restrict the number of user connections, and control access to
AMQP messaging resources by configuring _policies_.
+
+=== Types of Policies
+
+You can configure two different types of policies: _global policies_ and
_vhost policies_.
+
+Global policies::
+Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all vhost policies), and
defines how the router should use vhost policies.
+
+Vhost policies::
+Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or _vhost_). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
++
+[NOTE]
+
+A vhost is typically the name of the host to which the client connection
is directed. For example, if a client application opens a connection to the
`amqp://mybroker.example.com:5672/queue01` URL, the vhost would be
`mybroker.example.com`.
+
+
+The resource limits defined in global and vhost policies are applied to
user connections only. The limits do not affect inter-router connections or
router connections that are outbound to waypoints.
+
+=== How {RouterName} Applies Policies
+
+When a client connects to a router, the router determines whether to
permit the connection based on the global and vhost policies, and the following
properties of the connection:
+
+* The host to which the connection is directed (the vhost)
+* The connection's authenticated user name
+* The host from which the client is connecting (the remote host)
+
+If the connection is permitted, then the router applies a vhost policy
that matches the vhost to which the connection is directed. The vhost policy
limits are enforced for the lifetime of the connection.
+
+=== Configuring Global Policies
+
+You can set the incoming connection limit for the router and define how it
should use vhost policies by configuring a global policy.
+
+.Procedure
+
+* In the router configuration file, add a `policy` section.
++
+--
+[options="nowrap",subs="+quotes"]
+
+policy = {
+maxConnections: 1 // <1>
+enableVhostPolicy: true // <2>
+policyDir: /etc/qpid-dispatch/policies/ // <3>
+defaultVhost: $default // <4>
+}
+
+<1> The maximum number of concurrent client connections allowed for this
router. This limit is always enforced, even if no other policy settings have
been defined. The limit is applied to all incoming connections regardless of
remote host, authenticated user, or targeted vhost. The default value is
`65535`.
--- End diff --
maxConnections is restricted to be in the range 0..65535 inclusive. That
said, a setting of zero works but it renders the router to be useless as no
users or management clients can connect to it. The 65535 was chosen to be some
limit that is probably not achievable in the real world.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #262: Crolke 918 vhost hostname
Github user ChugR closed the pull request at: https://github.com/apache/qpid-dispatch/pull/262 --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #262: Crolke 918 vhost hostname
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/262 Close this PR and create a new one using @ganeshmurthy deprecationName --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #262: Crolke 918 vhost hostname
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/262#discussion_r171073403
--- Diff: console/config/schema.json ---
@@ -1273,6 +1273,10 @@
"type": "map",
"description": "A map where each key is a user group name and
the value is a map of the corresponding settings for that group."
},
+"hostname": {
+ "type": "string",
+ "description": "The vhost name."
+},
--- End diff --
will do.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #262: Crolke 918 vhost hostname
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/262#discussion_r170987746
--- Diff: console/config/schema.json ---
@@ -1273,6 +1273,10 @@
"type": "map",
"description": "A map where each key is a user group name and
the value is a map of the corresponding settings for that group."
},
+"hostname": {
--- End diff --
Can't be used until you check it in...
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #262: Crolke 918 vhost hostname
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/262 This PR adds vhost attribute 'hostname' as the name of the entity. It deprecates vhost attribute 'id'. Policy code warns if 'id' is used and fails if neither 'id' nor 'hostname' is specified. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #262: Crolke 918 vhost hostname
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/262 Crolke 918 vhost hostname You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch crolke-918-vhost-hostname Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/262.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #262 commit 09fd276746da848774a2c9314a4ea7fa7924545a Author: Chuck Rolke <crolke@...> Date: 2018-02-26T21:25:54Z DISPATCH-918: policy vhost 'id' renamed to 'hostname' commit 2078ab98da45d65a33a3613cac6e4c50261b1d6e Author: Chuck Rolke <crolke@...> Date: 2018-02-26T21:32:35Z DISPATCH-918: fix self test description --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #261: DISPATCH-918 - Initial modifications to Pyt...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/261#discussion_r170614842
--- Diff: tests/system_tests_autolinks.py ---
@@ -53,17 +53,18 @@ def setUpClass(cls):
#
# Create a pair of default auto-links for 'node.1'
#
-('autoLink', {'addr': 'node.1', 'containerId': 'container.1',
'dir': 'in'}),
-('autoLink', {'addr': 'node.1', 'containerId': 'container.1',
'dir': 'out'}),
+('autoLink', {'addr': 'node.1', 'containerId': 'container.1',
'direction': 'in'}),
+('autoLink', {'addr': 'node.1', 'containerId': 'container.1',
'direction': 'out'}),
#
# Create a pair of auto-links on non-default phases for
container-to-container transfers
#
-('autoLink', {'addr': 'xfer.2', 'containerId': 'container.2',
'dir': 'in', 'phase': '4'}),
-('autoLink', {'addr': 'xfer.2', 'containerId': 'container.3',
'dir': 'out', 'phase': '4'}),
+('autoLink', {'addr': 'xfer.2', 'containerId': 'container.2',
'direction': 'in', 'phase': '4'}),
+('autoLink', {'addr': 'xfer.2', 'containerId': 'container.3',
'direction': 'out', 'phase': '4'}),
#
# Create a pair of auto-links with a different external address
+# Leave the direction as dir to test backward compatibility.
--- End diff --
Is there a test that has both dir and direction specified to see that it
fails?
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #255: DISPATCH-333: Create new router policies do...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/255#discussion_r169471170
--- Diff: doc/new-book/configuration-security.adoc ---
@@ -412,3 +414,385 @@ listener {
For more information about these attributes, see
xref:adding_sasl_authentication_to_incoming_connection[].
--
+
+== Authorizing Access to Messaging Resources
+
+You can restrict the number of user connections, and control access to
AMQP messaging resources by configuring _policies_.
+
+=== Types of Policies
+
+You can configure two different types of policies: _global policies_ and
_vhost policies_.
+
+Global policies::
+Settings for the router. A global policy defines the maximum number of
incoming user connections for the router (across all vhost policies), and
defines how the router should use vhost policies.
+
+Vhost policies::
+Connection and AMQP resource limits for a messaging endpoint (called an
AMQP virtual host, or _vhost_). A vhost policy defines what a client can access
on a messaging endpoint over a particular connection.
++
+[NOTE]
+
+A vhost is typically the name of the host to which the client connection
is directed. For example, if a client application opens a connection to the
`amqp://mybroker.example.com:5672/queue01` URL, the vhost would be
`mybroker.example.com`.
+
+
+The resource limits defined in global and vhost policies are applied to
user connections only. The limits do not affect inter-router connections or
router connections that are outbound to waypoints.
+
+=== How {RouterName} Applies Policies
+
+When a client connects to a router, the router determines whether to
permit the connection based on the global and vhost policies, and the following
properties of the connection:
+
+* The host to which the connection is directed (the vhost)
+* The connection's authenticated user name
+* The host from which the client is connecting (the remote host)
+
+If the connection is permitted, then the router applies a vhost policy
that matches the vhost to which the connection is directed. The vhost policy
limits are enforced for the lifetime of the connection.
+
+=== Configuring Global Policies
+
+You can set the incoming connection limit for the router and define how it
should use vhost policies by configuring a global policy.
+
+.Procedure
+
+* In the router configuration file, add a `policy` section.
++
+--
+[options="nowrap",subs="+quotes"]
+
+policy = {
+maxConnections: _NUMBER_OF_CONNECTIONS_
+enableVhostPolicy: true | false
+policyDir: _PATH_
+defaultVhost: _VHOST_NAME_
+}
+
+
+`maxConnections`::
+The maximum number of concurrent client connections allowed for this
router. This limit is always enforced, even if no other policy settings have
been defined. The limit is applied to all incoming connections regardless of
remote host, authenticated user, or targeted vhost. The default value is
`65535`.
+
+`enableVhostPolicy`::
+Enables the router to enforce the connection denials and resource limits
defined in the configured vhost policies. The default is `false`, which means
that the router will not enforce any vhost policies.
++
+[NOTE]
+
+Setting `enableVhostPolicy` to `false` improves the router's performance.
+
+
+`policyDir`::
+The absolute path to a directory that holds vhost policy definition files
in JSON format (`*.json`). The router processes all of the vhost policies in
each JSON file that is in this directory. For more information, see
xref:configuring-vhost-policies-json[].
+
+`defaultVhost`::
+The name of the default vhost policy, which is applied to any connection
for which a vhost policy has not been configured. The default is `$default`. If
`defaultVhost` is not defined, then default vhost processing is disabled.
+--
+
+=== Configuring Vhost Policies
+
+You configure vhost policies to define the connection limits and AMQP
resource limits for a messaging endpoint.
+
+A vhost policy consists of the following:
+
+* Connection limits
++
+These limits control the number of users that can be connected to the
vhost simultaneously.
+
+* User groups
++
+A user group defines the messaging resources that the group members are
permitted to access. Each user group defines the following:
+
+** A set of users that can connect to the vhost (the group members)
+** The remote hosts from which the group members may connect to the router
network
+** The AMQP resources that the group members are permitted to access on
the vhost
+
+You can configure vhost policies directly in the router configuration
file, or
[GitHub] qpid-dispatch pull request #251: DISPATCH-920: Selftest exposing the issue
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/251 DISPATCH-920: Selftest exposing the issue You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch crolke-DISPATCH-920 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/251.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #251 commit 1f2e3d929af763300af9d901bbe14336ee72bcd5 Author: Chuck Rolke <crolke@...> Date: 2018-01-29T20:48:49Z DISPATCH-920: self test to detect interrouter links being blocked commit 6383909b7477bd5fefe64fbfa565f538b6d52f23 Author: Chuck Rolke <crolke@...> Date: 2018-01-29T21:02:37Z tidy up --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-proton issue #132: PROTON-1728: WIP: Reorganize the source tree and rem...
Github user ChugR commented on the issue: https://github.com/apache/qpid-proton/pull/132 * This seems like a great improvement overall. It's easier to find stuff without needing the folklore to locate it. * The presentation of the source tree gives the project a mature look and feel. * Compiled on windows VS2012 x64 and ran tests OK. A few observations: * Why is there a directory src/extra? This seems odd after such a massive cleanup. * As a downstream consumer of the binary artifacts the minor reorganization is acceptable. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #211: A pull request with Ganesh's, Chuck's, and ...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/211#discussion_r145424345
--- Diff: src/message_private.h ---
@@ -113,6 +113,7 @@ typedef struct {
bool receive_complete; // true if the
message has been completely received, false otherwise
sys_atomic_t fanout; // The number of
receivers for this message. This number does not include in-process subscribers.
bool q2_input_holdoff; // hold off
calling pn_link_recv
+bool aborted;// receive
completed with abort flag set
--- End diff --
For performance with smaller cache use it would make sense to group the
bools together and not split them with elements that have alignment
requirements.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #205: Dispatch 829 - reference only, do not pull ...
Github user ChugR closed the pull request at: https://github.com/apache/qpid-dispatch/pull/205 --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #208: Dispatch 829 3
Github user ChugR closed the pull request at: https://github.com/apache/qpid-dispatch/pull/208 --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #208: Dispatch 829 3
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/208 Dispatch 829 3 This adds to Ganesh's changes in PR #206 You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-829-3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/208.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #208 commit 4ea149a5ff0a854daa15a7ade16c29ca0636b0e2 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-28T14:35:45Z DISPATCH-829: Sense messages received with abort status. commit 87bef2ccf183ed8525c7f155b49af199dc8e1c3e Author: Chuck Rolke <cro...@redhat.com> Date: 2017-10-02T20:06:57Z DISPATCH-829: handle aborted messages commit 55690f22f33db05bc9a11238ef0cffdd128dee13 Author: Ganesh Murthy <gmur...@redhat.com> Date: 2017-10-05T21:59:01Z DISPATCH-829 - Added a proton_ref set flag so we can incref and decref accordingly commit 061fe4fb83ae9b8900741f45d573ce25e3ddfd04 Author: Ganesh Murthy <gmur...@redhat.com> Date: 2017-10-06T18:48:33Z DISPATCH-829 - Added code to abort delivery on link cleanup commit adbb876f59f593a159d75350521594693795a5a8 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-10-09T15:24:16Z DISPATCH-829: outbound aborted messages must be marked send_complete --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #205: Dispatch 829 - reference only, do not pull ...
GitHub user ChugR opened a pull request:
https://github.com/apache/qpid-dispatch/pull/205
Dispatch 829 - reference only, do not pull this code
Test setup:
* Proton master branch as of today.
* Dispatch conf:
# Router to run qpid-interop-test
router {
mode: interior
id: Router.A
workerThreads: 4
#allowUnsettledMulticast: yes
}
listener {
host: 0.0.0.0
port: 5672
authenticatePeer: no
saslMechanisms: ANONYMOUS
}
listener {
host: localhost
port: 5672
authenticatePeer: no
saslMechanisms: ANONYMOUS
}
address {
prefix: jms.queue.qpid-interop.#
distribution: balanced
}
log {
module: DEFAULT
enable: debug+
}
* Sender
proton/build/examples/c/send-abort "" 5672 jms.queue.qpid-interop.test 1
* Receiver
proton/build/examples/c/receive "" 5672 jms.queue.qpid-interop.test 1
* Result
qd_message_t being sent is never freed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-829-2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/qpid-dispatch/pull/205.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #205
commit 4ea149a5ff0a854daa15a7ade16c29ca0636b0e2
Author: Chuck Rolke <cro...@redhat.com>
Date: 2017-09-28T14:35:45Z
DISPATCH-829: Sense messages received with abort status.
commit da3303f4d1e65e0dfb89305fd4b5987bede5ccdf
Author: Chuck Rolke <cro...@redhat.com>
Date: 2017-10-02T20:06:57Z
DISPATCH-829: handle aborted messages
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #196: DISPATCH-807: Message handling requires flow contr...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/196 Merged through another branch --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #196: DISPATCH-807: Message handling requires flo...
Github user ChugR closed the pull request at: https://github.com/apache/qpid-dispatch/pull/196 --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #196: DISPATCH-807: Message handling requires flo...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/196 DISPATCH-807: Message handling requires flow control to limit memory consumption You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-807 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/196.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #196 commit f682283c2d9543ae121ea7aae9b3b4b900dd387d Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-14T13:53:55Z DISPATCH-807: Define limits, state, and accessors for Q2 input holdoff commit 224e2154ce2c3756bf628d007e3b85806395cd69 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-14T14:15:05Z DISPATCH-807: move q2 holdoff flag from message into content Random senders free message content buffers. Each sender must be able to sense the receiver's holdoff state to know when to schedule a callback. Message content is common to the receiver and to all the senders. commit 14e1a13b33a2fecea0c8bc0f93c4c402094e8dc7 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-14T14:33:50Z DISPATCH-807: Remove delivery arg from delivery_handler interface Container delivery handler has a qd_link arg and from that the pn_link and the current pn_delivery may be derived. commit 131dfa49c56cc80a1cd20c3461defafd6b710caf Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-14T20:01:32Z DISPATCH-807: Implement Q2 throttling * Add container accessor for link->node->context * qd_message_send signals when enough room for q2 resume is available * Add message accessor for receiving qd_link * Promote server connection_wake to public * Use macros to name LOCK, UNLOCK functions for legibility * Sense and set q2 holdoff in message_receive * Sense and clear q2 holdoff in message_send * Add deferred callback for AMQP_rx_handler commit 62a7613fbae245c337b204ee86e5b1958afaa987 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-14T20:47:33Z DISPATCH-807: remove asserts for items that disappear regularly commit 998de77a8abb754f96f4f3d1c941f2562521537f Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-14T21:00:09Z DISPATCH-807: Implement Q3 limits based on send session outgoing bytes commit 63526470733f54746b93051e4aaae88815c2a20b Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-15T14:01:54Z DISPATCH-807: Adjust sizes of q2 limits --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #194: DISPATCH-825: Fix interlocking between mess...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/194#discussion_r138710888
--- Diff: src/message.c ---
@@ -1109,95 +1146,97 @@ qd_message_t *qd_message_receive(pn_delivery_t
*delivery)
}
//
-// The discard flag indicates if we should continue receiving the
message.
-// This is pertinent in the case of large messages. When large
messages are being received, we try to send out part of the
-// message that has been received so far. If we not able to send it
anywhere, there is no need to keep creating buffers
-//
-bool discard = qd_message_is_discard((qd_message_t*)msg);
-
-//
-// Get a reference to the tail buffer on the message. This is the
buffer into which
-// we will store incoming message data. If there is no buffer in the
message, this is the
-// first time we are here and we need to allocate an empty one and add
it to the message.
+// The discard flag indicates we should keep reading the input stream
+// but not process the message for delivery.
//
-if (!discard) {
-buf = DEQ_TAIL(msg->content->buffers);
-if (!buf) {
-buf = qd_buffer();
-DEQ_INSERT_TAIL(msg->content->buffers, buf);
-}
+if (qd_message_is_discard((qd_message_t*)msg)) {
+return discard_receive(delivery, link, (qd_message_t *)msg);
}
+
+// Loop until msg is complete, error seen, or incoming bytes are
consumed
+bool recv_error = false;
while (1) {
-if (discard) {
-char dummy[BUFFER_SIZE];
-rc = pn_link_recv(link, dummy, BUFFER_SIZE);
-}
-else {
-//
-// Try to receive enough data to fill the remaining space in
the tail buffer.
-//
+//
+// handle EOS and clean up after pn receive errors
+//
+bool at_eos = (pn_delivery_partial(delivery) == false) &&
+ (pn_delivery_pending(delivery) == 0);
+
+if (at_eos || recv_error) {
+// Message is complete
+sys_mutex_lock(msg->content->lock);
+{
+// Append last buffer if any with data
+if (msg->content->pending) {
+if (qd_buffer_size(msg->content->pending) > 0) {
+// pending buffer has bytes that are port of
message
+DEQ_INSERT_TAIL(msg->content->buffers,
+msg->content->pending);
+} else {
+// pending buffer is empty
+qd_buffer_free(msg->content->pending);
+}
+msg->content->pending = 0;
+} else {
+// pending buffer is absent
+}
+
+msg->content->receive_complete = true;
-rc = pn_link_recv(link, (char*) qd_buffer_cursor(buf),
qd_buffer_capacity(buf));
+// unlink message and delivery
+pn_record_set(record, PN_DELIVERY_CTX, 0);
+}
+sys_mutex_unlock(msg->content->lock);
+return (qd_message_t*) msg;
}
//
-// If we receive PN_EOS, we have come to the end of the message.
+// Handle a missing or full pending buffer
//
-if (rc == PN_EOS) {
-//
-// We have received the entire message since rc == PN_EOS, set
the receive_complete flag to true
-//
-msg->content->receive_complete = true;
-
-//
-// Clear the value in the record with key PN_DELIVERY_CTX
-//
-pn_record_set(record, PN_DELIVERY_CTX, 0);
-
-//
-// If the last buffer in the list is empty, remove it and free
it. This
-// will only happen if the size of the message content is an
exact multiple
-// of the buffer size.
-//
-if (buf && qd_buffer_size(buf) == 0) {
+if (!msg->content->pending) {
+// Pending buffer is absent: get a new one
+msg->content->pending = qd_buffer();
+} else {
+// Pending buffer exists
+if (qd_buffer_capacity(msg->content->pending) == 0) {
+// Pending buffer is full
sys_mutex_lock(msg->content->lock);
-
[GitHub] qpid-dispatch pull request #194: DISPATCH-825: Fix interlocking between mess...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/194 DISPATCH-825: Fix interlocking between message send and receive Don't allow a buffer on the message buffer chain that might be removed. Use a content-based pending buffer instead. Don't release message lock between adding final buffer and setting receive_complete flag. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-825 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/194.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #194 commit b6e10668a59fc3953698ce1fcc27f45b307276c3 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-09-13T15:49:30Z DISPATCH-825: Fix interlocking between message send and receive Don't allow a buffer on the message buffer chain that might be removed. Use a content-based pending buffer instead. Don't release message lock between adding final buffer and setting receive_complete flag. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #177: DISPATCH-767 - Added code to support multi ...
Github user ChugR commented on a diff in the pull request: https://github.com/apache/qpid-dispatch/pull/177#discussion_r129305989 --- Diff: src/message.c --- @@ -846,6 +853,8 @@ qd_message_t *qd_message() } ZERO(msg->content); +msg->content->receive_complete = false; +msg->content->discard = false; --- End diff -- after ZEROing the content setting any value to 0 or false (seven places) is a waste. Maybe change the assignments to asserts if you don't trust the ZERO. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #177: DISPATCH-767 - Added code to support multi ...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/177#discussion_r129307766
--- Diff: src/message.c ---
@@ -999,11 +1014,84 @@ void qd_message_set_ingress_annotation(qd_message_t
*in_msg, qd_composed_field_t
qd_compose_free(ingress_field);
}
+bool qd_message_is_discard(qd_message_t *msg)
+{
+if (!msg)
+return false;
+qd_message_pvt_t *pvt_msg = (qd_message_pvt_t*) msg;
+return pvt_msg->content->discard;
+}
+
+void qd_message_set_discard(qd_message_t *msg, bool discard)
+{
+if (!msg)
+return;
+
+qd_message_pvt_t *pvt_msg = (qd_message_pvt_t*) msg;
+pvt_msg->content->discard = discard;
+}
+
+size_t qd_message_fanout(qd_message_t *in_msg)
+{
+if (!in_msg)
+return 0;
+qd_message_pvt_t *msg = (qd_message_pvt_t*) in_msg;
+return msg->content->fanout;
+}
+
+void qd_message_add_fanout(qd_message_t *in_msg)
+{
+assert(in_msg);
+qd_message_pvt_t *msg = (qd_message_pvt_t*) in_msg;
+msg->content->fanout++;
--- End diff --
atomic inc?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #177: DISPATCH-767 - Added code to support multi ...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/177#discussion_r129308722
--- Diff: src/message.c ---
@@ -999,11 +1014,84 @@ void qd_message_set_ingress_annotation(qd_message_t
*in_msg, qd_composed_field_t
qd_compose_free(ingress_field);
}
+bool qd_message_is_discard(qd_message_t *msg)
+{
+if (!msg)
+return false;
+qd_message_pvt_t *pvt_msg = (qd_message_pvt_t*) msg;
+return pvt_msg->content->discard;
+}
+
+void qd_message_set_discard(qd_message_t *msg, bool discard)
+{
+if (!msg)
+return;
+
+qd_message_pvt_t *pvt_msg = (qd_message_pvt_t*) msg;
+pvt_msg->content->discard = discard;
+}
+
+size_t qd_message_fanout(qd_message_t *in_msg)
+{
+if (!in_msg)
+return 0;
+qd_message_pvt_t *msg = (qd_message_pvt_t*) in_msg;
+return msg->content->fanout;
+}
+
+void qd_message_add_fanout(qd_message_t *in_msg)
+{
+assert(in_msg);
+qd_message_pvt_t *msg = (qd_message_pvt_t*) in_msg;
+msg->content->fanout++;
+}
+
+bool qd_message_receive_complete(qd_message_t *in_msg)
+{
+if (!in_msg)
+return false;
+qd_message_pvt_t *msg = (qd_message_pvt_t*) in_msg;
+return msg->content->receive_complete;
+}
+
+bool qd_message_send_complete(qd_message_t *in_msg)
+{
+if (!in_msg)
+return false;
+
+qd_message_pvt_t *msg = (qd_message_pvt_t*) in_msg;
+return msg->send_complete;
+}
+
+bool qd_message_tag_sent(qd_message_t *in_msg)
+{
+if (!in_msg)
+return false;
+
+qd_message_pvt_t *msg = (qd_message_pvt_t*) in_msg;
+return msg->tag_sent;
+}
+
+void qd_message_set_tag_sent(qd_message_t *in_msg, bool tag_sent)
+{
+if (!in_msg)
+return;
+
+qd_message_pvt_t *msg = (qd_message_pvt_t*) in_msg;
+msg->tag_sent = tag_sent;
+}
+
+qd_field_location_t qd_message_cursor(qd_message_pvt_t *in_msg)
--- End diff --
prefer a 'qd_field_location_t *' return and not a copy of the whole struct
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #177: DISPATCH-767 - Added code to support multi ...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/177#discussion_r129318215
--- Diff: src/message_private.h ---
@@ -107,11 +106,21 @@ typedef struct {
qd_parsed_field_t *ma_pf_to_override;
qd_parsed_field_t *ma_pf_trace;
int ma_int_phase;
+//qd_parsed_field_t *parsed_message_annotations;
+
+bool discard;// Should this
message be discarded?
+bool receive_complete; // true if the
message has been completely received, false otherwise
+unsigned int fanout; // The number of
receivers for this message. This number does not include in-process subscribers.
} qd_message_content_t;
typedef struct {
DEQ_LINKS(qd_message_t); // Deque linkage that overlays the
qd_message_t
-qd_message_content_t *content;
+qd_field_location_t cursor; // A pointer to the current
location of the outgoing byte stream.
--- End diff --
qd_iterator_pointer_t might be more appropriate for a pointer into buffers.
field_location has baggage that is not needed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #177: DISPATCH-767 - Added code to support multi ...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/177#discussion_r129306967
--- Diff: src/buffer.c ---
@@ -83,7 +85,26 @@ size_t qd_buffer_size(qd_buffer_t *buf)
void qd_buffer_insert(qd_buffer_t *buf, size_t len)
{
buf->size += len;
-assert(buf->size <= buffer_size);
+assert(buf->size <= BUFFER_SIZE);
+}
+
+void qd_buffer_add_fanout(qd_buffer_t *buf)
+{
+buf->fanout++;
--- End diff --
Is this called with a lock held? Otherwise probably needs to be atomic
increment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #177: DISPATCH-767 - Added code to support multi ...
Github user ChugR commented on a diff in the pull request: https://github.com/apache/qpid-dispatch/pull/177#discussion_r129306348 --- Diff: include/qpid/dispatch/message.h --- @@ -296,6 +296,61 @@ qd_parsed_field_t *qd_message_get_trace (qd_message_t *msg); */ intqd_message_get_phase_val (qd_message_t *msg); +/* + * Should the message be discarded. + * A message can be discarded if the disposition is released or rejected. + * + * @param msg A pointer to the message. + **/ +bool qd_message_is_discard(qd_message_t *msg); + +/** + *Set the discard field on the message to to the passed in boolean value. + * + * @param msg A pointer to the message. + * @param discard - the boolean value of discard. + */ +void qd_message_set_discard(qd_message_t *msg, bool discard); + +/** + * Has the message been completely received? + * Return true if the message is fully received + * Returns false if only the partial message has been received, if there is more of the message to be received. + * + * @param msg A pointer to the message. + */ +bool qd_message_receive_complete(qd_message_t *msg); + +/** + * Returns true if the message has been completely received AND the message has been completely sent. + */ +bool qd_message_send_complete(qd_message_t *msg); + +/** + * Returns true if the delivery tag has already been sent. + */ +bool qd_message_tag_sent(qd_message_t *msg); + + +/** + * Returns true if the delivery tag has already been sent. --- End diff -- cut-paste issue. This function sets something and returns nothing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #175: DISPATCH-759: Move freeing message and iter...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/175 DISPATCH-759: Move freeing message and iterator out of core thread Some of delivery deletion requires core context and that part stays. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch DISPATCH-759 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/175.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #175 commit a32a4e556b41a998ee5b0b558688919d6e536d92 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-08T14:25:24Z DISPATCH-759: Move freeing message and iterator out of core thread Some of delivery deletion requires core context and that part stays. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #173: DISPATCH-760: Message annotation handling i...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/173 DISPATCH-760: Message annotation handling improvements This PR replaces existing annotation handling with a more efficient version. It is fully compatible with 0.8.x. It contains the same stuff as [pr171](https://github.com/apache/qpid-dispatch/pull/171) with the addition of five commits (b321fa2..50637b7) to clean up and satisfy review comments A comparison with 0.8.x has run on a single laptop. A sender sends a short message with a varying number of 10-byte user annotations to an ingress router A. Router A sends the messages to egress router B. A single client is attached to router B. This comparison is not rigorous and at the 50-annotation-per-message level the producer started to be the limiting factor. Messages per second. Number of annotations, Nanno, vs router code version, 0.8.x or new. | Nanno | 0.8.x | new | |--|---|| | 0 |26500 | 32000 | | 1 | 24000 | 31200 | | 5 |18000 | 30100 | | 10 |14200 | 28500 | | 20 |8200 | 24400 | | 50 |3500 | 11300 | This PR performance numbers are down a little bit from the last PR. This PR does a little extra work to prevent router annotations from being spoofed by user annotations. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch crolke-DISPATCH-760-3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/173.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #173 commit 1c3b0dd6f2e64718242c3da152ae56312ab2d365 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T15:04:17Z DISPATCH-760: accessor for connection status: interrouter vs user commit 6e8b56e5b5b7b2a5ea1e65b34c2f93c885bbb599 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T15:46:43Z DISPATCH-760: add opaque compose map item insertion functions commit cf2c76a924de2a487a92d294b2f2de39506a30ee Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T15:56:08Z DISPATCH-760: promote pointer_t to public qd_iterator_pointer_t commit 7f650d8f9f3106942201b0b6ddc7eb4fe434a9a9 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T16:30:35Z DISPATCH-760: Add prefix search functions that use iterator pointers commit b67b2013fd6c666c80119704821b22c8ee431257 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T20:57:17Z DISPATCH-760: Add new annotation parser commit 21014c6e9f8dd2ea05551f341487a9d486c074cd Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-01T19:01:56Z DISPATCH-760: Message handling uses new scheme commit 3b05873fcd0f04fda3682e42b024be0ff60babca Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-01T19:51:00Z DISPATCH-760: compare key names using strcmp not iter_equal commit 95a05563a71cba7a46ce5f462a89e4f6b0e610d7 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-03T15:23:39Z DISPATCH-760: Get self tests going Get rid of is_interrouter flag in favor of existing strip_annotations flags. Pass strip_annotations_in through message to parse function. commit cd7d5128ea9ece90ce1236ad027f5c4f448656df Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-03T16:05:36Z DISPATCH-760: Fix inbound annotation stripping commit b321fa26579470107fbeadc3815488155db273fd Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-05T21:45:06Z DISPATCH-760: don't re-strip inbound annotations commit 40cb7c1ef8811b1a948f9233cd99e128d3089c15 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-06T11:03:16Z DISPATCH-760: improve annotation V0 logic commit 425c898f623b973d30e98e1e971c70faff944da0 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-06T13:03:41Z DISPATCH-760: Prevent user anno's from being considered in routing logic Always provide four routing annotations by padding if necessary. commit 9672b71912f50a18df0ce392e8775077336b5553 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-06T17:16:03Z DISPATCH-760: remove strip flag The flag is only set false and is not needed. commit 50637b770400e106bfb1c2e2ad7c57f8f8b2321e Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-06T20:14:35Z DISPATCH-760: Change magic numbers into formal constants Router message annotation uses some of map keys and will at most use QD_MA_N_KEYS annotation entries. The longest key strlen is QD_MA_MAX_KEY_LEN bytes. Router message annotation keys are at the end of the annotation section. Code uses a sliding window to avoid searching
[GitHub] qpid-dispatch pull request #171: DISPATCH-760: Message annotation handling i...
Github user ChugR closed the pull request at: https://github.com/apache/qpid-dispatch/pull/171 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #171: DISPATCH-760: Message annotation handling i...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/171 DISPATCH-760: Message annotation handling improvements This PR replaces existing annotation handling with a more efficient version. It is fully compatible with 0.8.x. A comparison with 0.8.x has run on a single laptop. A sender sends a short message with a varying number of 10-byte user annotations to an ingress router A. Router A sends the messages to egress router B. A single client is attached to router B. This comparison is not rigorous and at the 50-annotation-per-message level the producer started to be the limiting factor. Messages per second. Number of annotations, Nanno, vs router code version, 0.8.x or new. | Nanno | 0.8.x | new | |--|---|| | 0 |26500 | 33700 | | 1 | 24000 | 33100 | | 5 |18000 | 31300 | | 10 |14200 | 29700 | | 20 |8200| 25200 | | 50 |3500 | 11700 | You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch crolke-DISPATCH-760-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/171.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #171 commit 3e2809f36130f210d0baa8391ca8d3ab4a3d6851 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T15:04:17Z DISPATCH-760: accessor for connection status: interrouter vs user commit f3707789d7fa5c598bc564b47d47f1893f74c495 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T15:46:43Z DISPATCH-760: add opaque compose map item insertion functions commit f24bf97d5a90f278a7f1700bc0c0d109094fbc06 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T15:56:08Z DISPATCH-760: promote pointer_t to public qd_iterator_pointer_t commit a9d0aff33a7416d9bf6640abce11a3b4080758ec Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T16:30:35Z DISPATCH-760: Add prefix search functions that use iterator pointers commit 4698f2fe226506d78d39bb31cfbfa853d876ad9a Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-30T20:57:17Z DISPATCH-760: Add new annotation parser commit feaf1ce1bf5ba69bed4c59c9928e19415da28577 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-01T19:01:56Z DISPATCH-760: Message handling uses new scheme commit 6de0c999cc00affe57cd82410692ab2f824acf02 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-01T19:51:00Z DISPATCH-760: compare key names using strcmp not iter_equal commit 162ae8db5c434017181c13c009a2c18fcd653d0a Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-03T15:23:39Z DISPATCH-760: Get self tests going Get rid of is_interrouter flag in favor of existing strip_annotations flags. Pass strip_annotations_in through message to parse function. commit aaeb03006d136d3dc09e004b07cb2285906e8f3c Author: Chuck Rolke <cro...@redhat.com> Date: 2017-07-03T16:05:36Z DISPATCH-760: Fix inbound annotation stripping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #168: crolke dev annotations - work in progress P...
Github user ChugR closed the pull request at: https://github.com/apache/qpid-dispatch/pull/168 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #168: crolke dev annotations - work in progress PR for r...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/168 The straight new V2 method offers about a 35% message rate improvement over the existing method that shipped in 0.8.x. However, the new way of doing the original method offers a 25% improvement. I'm withdrawing this PR and working on better processing of "the old way". Keeping the old annotations eliminates incompatible versions V1 vs V2, reduces code clutter, eliminates peer version detection, and simplifies testing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch issue #168: crolke dev annotations - work in progress PR for r...
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/168 Well spotted. That's a mistake from a failed merge conflict. Will fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #168: crolke dev annotations - work in progress P...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/168 crolke dev annotations - work in progress PR for review **DO NOT COMMIT THIS PULL REQUEST** It is for review only. This work adds a new annotation scheme for inter-router annotations. The old method put a variable number (0..4) of key-value pairs into the message annotations map at the end of the map. The new method puts in a single list of four elements at the beginning of the message annotations map. In order to be backwards compatible there is some amount of version sensing and translating between annotation formats. This could be improved in a number of ways. This commit still fails two self tests: system_tests_one_router and system_tests_two_routers in their annotation handling sections. The V1 vs V2 annotation feature is not tested much at all. That is in the works after the general strategy is approved. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch crolke-dev-annotations Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/168.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #168 commit b3018514a09419f2bec669b50c7ac9178d8db5b4 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-05-18T19:45:39Z Add v2 annotation message variables commit 3f6fef7075456034c28e6ad5c29f9203db8d37c0 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-05-18T19:49:32Z fix typo in comment commit eb87c3a99ddece5545e93be14ba875d2c00566dd Author: Chuck Rolke <cro...@redhat.com> Date: 2017-05-18T19:50:06Z add v2 annotation map key string commit d050adc61c4d6682b569ab6c3a3eca715e3ad0a4 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-05-22T20:05:26Z Slice router annotations from head of incoming message annotations. Reorganize data fields in message content. commit 5d440a385338427eb8c30132e0cc3cb3db0c0d4f Author: Chuck Rolke <cro...@redhat.com> Date: 2017-05-24T22:09:31Z Further progress on new annatation scheme Add accessors for private content fields Add compose_annotations Add router_annotate_message Add AMWQP_rx_handler commit c69304180466535ebabc9c74c6a4436a6a5ea731 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-05-26T21:36:18Z Get new scheme limping along. ANNO v2 runs at 1.54x over v1 with no annotations. Still has an issue with user annotations. commit c11b667a7b3040883a98e8cb7f2b556018172c17 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-01T14:14:10Z Running test code with new scheme still has self-test case fails and no new tests send delivery annotations if present. Future jira Fix size vs capacity bug parse annotations after copy commit 6ecdd9585e378a59960ed4618bc5114e378ce99f Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-01T17:18:08Z Bugfix: don't clobber the cursor commit ba1f7239c471dc685417a060e5709cd51236b5ef Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-01T20:20:07Z Avoid making copies of user annotations. Instead refer to them in-place in original received message. commit da89a162306c788f1ab278e46f07999759182d9e Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-01T20:36:42Z Improve comments commit eddcf103e4d1dc61735c948ef840105be658f9a4 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-01T20:42:23Z Remove debug scaffolding commit 460c51ba40b12c248bafb74b53b86c8cc1ddcd8e Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-05T17:25:36Z Run either message annotation scheme based on sensed version. Always choose v2 unless the connection is to a router running version "0*" code. commit 7f9680435ddc45eb17cea0f556d68748facfd531 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-05T20:09:39Z Fix annotation tests to use new scheme. The old annotations are switched out. TODO: Make annotation scheme testable for all variations of old and new both going in and coming out. commit 19a0f3ddbeb7230194c833e7446883766c22198e Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-05T21:02:58Z Tidy up one- and two-router annotation tests commit f4b35f0cc274e5c137566aa57e5d4ccc93598ea8 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-06-20T17:41:59Z Restructure the new feature more carefully All tests pass except system_test_one_router and _two_routers. Time for some review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i
[GitHub] qpid-dispatch issue #159: Metrics support
Github user ChugR commented on the issue: https://github.com/apache/qpid-dispatch/pull/159 This patch adds a competing statistic for one that already exists. Going forward it would require a completely redundant set and every statistic would be counted twice. That's not desirable for the long run. Another approach might be to hook up the http-libwebsockets to the existing statistics and then to export them in the desired format. The existing management database has configuration data and runtime statistics together. See https://github.com/apache/qpid-dispatch/blob/master/python/qpid_dispatch/management/qdrouter.json.readme.txt This would be more work to hook up the the metric_manager as the code would have to hunt for all the values where they exist today. But in the end the resulting reports would have identical descriptions and values reported by qdstat and the other management tools. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #155: Remove unreachable code.
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/155 Remove unreachable code. Early in this function the field is limited to indicate something in the Properties section. `assert(QD_FIELD_MESSAGE_ID <= field && field <= QD_FIELD_REPLY_TO_GROUP_ID); ` Now the field may not be QD_FIELD_PROPERTIES. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch private_crolke_message_field_unreachable Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/155.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #155 commit 56ab5020e793b3a215c77dec79c97fa55745f1f5 Author: Chuck Rolke <cro...@redhat.com> Date: 2017-03-27T21:19:58Z Remove unreachable code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #106: DISPATCH-451: Configurable settings for ses...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/106#discussion_r81838943
--- Diff: include/qpid/dispatch/server.h ---
@@ -418,6 +418,22 @@ typedef struct qd_server_config_t {
uint32_t max_frame_size;
/**
+ * The max_sessions value is the number of sessions allowed on the
Connection.
+ * This value minus one is the Open performative channel-max setting.
+ */
+uint32_t max_sessions;
+
+/**
+ * The max_session_window value is the maximum number of outstanding
octets that are
--- End diff --
Good suggestion. The setting down to proton would be *
max-frame-size octets.
There is an identical setting in policy that operates this way, too. Again,
they ought to be the same either way.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #106: DISPATCH-451: Configurable settings for ses...
Github user ChugR commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/106#discussion_r81838905
--- Diff: include/qpid/dispatch/server.h ---
@@ -418,6 +418,22 @@ typedef struct qd_server_config_t {
uint32_t max_frame_size;
/**
+ * The max_sessions value is the number of sessions allowed on the
Connection.
+ * This value minus one is the Open performative channel-max setting.
+ */
+uint32_t max_sessions;
--- End diff --
There should be a comment that says if you specify zero then you get the
default number. Using channel_max is awkward in its own way: if the setting is
1 then you get 2.
There is an identical setting in policy that already used the 1..N
numbering (and not 0..N-1). They ought to be the same either way.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #106: DISPATCH-451: Configurable settings for ses...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/106 DISPATCH-451: Configurable settings for session capacity and max sessions This patch allows listener and connector settings for session capacity and for max sessions. It does _not_ allow a setting for max-message-size. Proton does not provide an interface for changing max message size. This issue is tracked at [PROTON-1311](https://issues.apache.org/jira/browse/PROTON-1311) Work is progressing on self tests for this feature. Proton python and wrapper interfaces tend to hide sessions and make finding session state difficult. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch crolke-DISPATCH-451 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/106.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #106 commit 5f4e4a6f2a0dc326bf51823036ab8e6bad9df063 Author: Chuck Rolke <cro...@redhat.com> Date: 2016-10-03T17:48:52Z DISPATCH-451: Allow configurable maxSessions and maxSessionWindow commit c248a34f3d2da46bb7e7e67ccbd04e2340b69f15 Author: Chuck Rolke <cro...@redhat.com> Date: 2016-10-03T18:37:40Z DISPATCH-451: Describe policy setting priority over listener/connector setting --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request #97: DISPATCH-468: Use computed x.509 cert displa...
GitHub user ChugR opened a pull request: https://github.com/apache/qpid-dispatch/pull/97 DISPATCH-468: Use computed x.509 cert display-name as authenticated id Add a DisplayName service interface to be called from C code. During connection establishment use the translated display name as the connection authenticated user_id. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ChugR/qpid-dispatch crolke-DISPATCH-468 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/qpid-dispatch/pull/97.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #97 commit 1bafd8b3a78da952ccd20c163104bcfb89f6155a Author: Chuck Rolke <cro...@redhat.com> Date: 2016-09-07T21:42:27Z DISPATCH-468: Use computed x.509 cert display-name as authenticated id Add a DisplayName service interface to be called from C code. During connection establishment use the translated display name as the connection authenticated user_id. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request: DISPATCH-163 - Added username and pass...
Github user ChugR commented on the pull request: https://github.com/apache/qpid-dispatch/pull/21#issuecomment-149681423 I tried this new branch and I think it's better for tests (*.py) all to end up in the /tests folder. At first the test failed. The problem was that the *.pyc files are all in the /tests folder and there was a stale system_tests_sasl_plain.pyc that was running. If all the .py files are in the build folder then the source is pristine and the build artifacts from one build are not interfering with another build. Either way the new test passes for me on Fedora 21. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid-dispatch pull request: DISPATCH-163 - Added username and pass...
Github user ChugR commented on the pull request: https://github.com/apache/qpid-dispatch/pull/19#issuecomment-148835284 How do we manage these credentials internally? Or when viewed through the management interface? Will 'qdmanage query' show the credentials in plain text? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] qpid pull request: QPID-6213: fix queue cleaner connection timeout...
Github user ChugR closed the pull request at: https://github.com/apache/qpid/pull/4 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
