[GitHub] [qpid-dispatch] ted-ross commented on a change in pull request #560: NO-JIRA: add a document explaining the router's threading implementat…
ted-ross commented on a change in pull request #560: NO-JIRA: add a document explaining the router's threading implementat… URL: https://github.com/apache/qpid-dispatch/pull/560#discussion_r367990164 ## File path: docs/notes/threading-info.txt ## @@ -0,0 +1,563 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + + += +Router Threading and Interprocess Communication Guide += + +The Qpid Dispatch Router (qdrouterd) threading architecture is based +on two classes of threads: + + *) A Worker thread which interfaces with the Proton subsystem, and + *) The Core Routing thread which manages the routing database and +performs forwarding. + +In a running router there is a single Core Routing thread (core) and +one or more Worker threads. The number of Worker threads is +determined by the router configuration setting "workerThreads", which +defaults to four. + +IPC between these threads is done using Action queues, Work queues, +and manipulation of shared data. + + +Worker Threads +== + +The Worker thread is responsible for interfacing with the Proton +subsystem. Only a worker thread can safely call directly into the +Proton library. The core thread must communicate with a worker thread +in order to have the worker manipulate Proton state on the core's +behalf. + +The Proton subsystem limits concurrency to a single connection. That +is, only one thread can be processing a given Proton connection (and +all of its child elements, such as links and deliveries) at a time. +The router honors this requirement by restricting access to a given +Proton connection (and its children) to a single worker thread at a +time. To say this another way, a particular Proton connection can be +processed by any worker threads but not concurrently. + +A worker thread is driven by the Proton proactor API. The worker's +main loop blocks on the proactor waiting for events, processes +incoming events, then blocks again. + + +Core Thread +=== + +The one core thread has several responsibilities, including: + + *) Managing the forwarding state + *) Forwarding messages across the router + *) Forwarding disposition and settlement state changes across the router + *) Managing link credit flow + *) Running the management Agent + +The core thread can be thought of as sitting in between the worker +threads, moving messages and state between them. + +When a worker thread needs to forward a received message it passes the +message and associated delivery state to the core thread. The core +thread uses address information from the message to determine the +outgoing link(s) for the message. The core then queues the message to +the proper outgoing link(s). The core wakes the worker thread(s) (via +the Proton proactor) so the message(s) can be written out the Proton +link. + +When delivery disposition or settlement changes are detected by a +worker thread it notifies the core thread. The core thread then +finds the link or delivery to which the changes need to be propagated +to. This results in the core thread setting the new state in the +link/delivery and waking a worker thread to update the new state in +Proton. + +The core also manages credit flow. The core grants credit to inbound +links. The core grants an initial batch of credit to a non-anonymous +link (a link with a target address) when the target address is +available for routing. The core will continue granting credit to the +link as long as the address is routable. The core ties the +replenishment of credit with the settlement of messages arriving on +the link: when the message is settled a single credit is granted by +the core. All credit flow operations involve having the core put a +credit flow work event on the proper inbound link then waking the +worker thread to update the credit in proton. + +The core can be the destination or source of AMQP message flows +from/to clients outside the router. This is used by services like the +Management Agent to receive management requests and send responses to +management clients. The core sends and receives these messages in +exactly the same
[GitHub] [qpid-dispatch] ted-ross commented on a change in pull request #560: NO-JIRA: add a document explaining the router's threading implementat…
ted-ross commented on a change in pull request #560: NO-JIRA: add a document explaining the router's threading implementat… URL: https://github.com/apache/qpid-dispatch/pull/560#discussion_r367988503 ## File path: docs/notes/threading-info.txt ## @@ -0,0 +1,563 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + + += +Router Threading and Interprocess Communication Guide += + +The Qpid Dispatch Router (qdrouterd) threading architecture is based +on two classes of threads: + + *) A Worker thread which interfaces with the Proton subsystem, and + *) The Core Routing thread which manages the routing database and +performs forwarding. + +In a running router there is a single Core Routing thread (core) and +one or more Worker threads. The number of Worker threads is +determined by the router configuration setting "workerThreads", which +defaults to four. + +IPC between these threads is done using Action queues, Work queues, +and manipulation of shared data. + + +Worker Threads +== + +The Worker thread is responsible for interfacing with the Proton +subsystem. Only a worker thread can safely call directly into the +Proton library. The core thread must communicate with a worker thread +in order to have the worker manipulate Proton state on the core's +behalf. + +The Proton subsystem limits concurrency to a single connection. That +is, only one thread can be processing a given Proton connection (and +all of its child elements, such as links and deliveries) at a time. +The router honors this requirement by restricting access to a given +Proton connection (and its children) to a single worker thread at a +time. To say this another way, a particular Proton connection can be +processed by any worker threads but not concurrently. + +A worker thread is driven by the Proton proactor API. The worker's +main loop blocks on the proactor waiting for events, processes +incoming events, then blocks again. + + +Core Thread +=== + +The one core thread has several responsibilities, including: + + *) Managing the forwarding state + *) Forwarding messages across the router + *) Forwarding disposition and settlement state changes across the router + *) Managing link credit flow + *) Running the management Agent + +The core thread can be thought of as sitting in between the worker +threads, moving messages and state between them. + +When a worker thread needs to forward a received message it passes the +message and associated delivery state to the core thread. The core +thread uses address information from the message to determine the +outgoing link(s) for the message. The core then queues the message to +the proper outgoing link(s). The core wakes the worker thread(s) (via +the Proton proactor) so the message(s) can be written out the Proton +link. + +When delivery disposition or settlement changes are detected by a +worker thread it notifies the core thread. The core thread then +finds the link or delivery to which the changes need to be propagated +to. This results in the core thread setting the new state in the +link/delivery and waking a worker thread to update the new state in +Proton. + +The core also manages credit flow. The core grants credit to inbound +links. The core grants an initial batch of credit to a non-anonymous +link (a link with a target address) when the target address is +available for routing. The core will continue granting credit to the +link as long as the address is routable. The core ties the +replenishment of credit with the settlement of messages arriving on +the link: when the message is settled a single credit is granted by +the core. All credit flow operations involve having the core put a +credit flow work event on the proper inbound link then waking the +worker thread to update the credit in proton. + +The core can be the destination or source of AMQP message flows +from/to clients outside the router. This is used by services like the +Management Agent to receive management requests and send responses to +management clients. The core sends and receives these messages in +exactly the same
[GitHub] [qpid-dispatch] ted-ross commented on a change in pull request #560: NO-JIRA: add a document explaining the router's threading implementat…
ted-ross commented on a change in pull request #560: NO-JIRA: add a document explaining the router's threading implementat… URL: https://github.com/apache/qpid-dispatch/pull/560#discussion_r367986840 ## File path: docs/notes/threading-info.txt ## @@ -0,0 +1,563 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + + += +Router Threading and Interprocess Communication Guide += + +The Qpid Dispatch Router (qdrouterd) threading architecture is based +on two classes of threads: + + *) A Worker thread which interfaces with the Proton subsystem, and + *) The Core Routing thread which manages the routing database and +performs forwarding. + +In a running router there is a single Core Routing thread (core) and +one or more Worker threads. The number of Worker threads is +determined by the router configuration setting "workerThreads", which +defaults to four. + +IPC between these threads is done using Action queues, Work queues, +and manipulation of shared data. + + +Worker Threads +== + +The Worker thread is responsible for interfacing with the Proton +subsystem. Only a worker thread can safely call directly into the +Proton library. The core thread must communicate with a worker thread +in order to have the worker manipulate Proton state on the core's +behalf. + +The Proton subsystem limits concurrency to a single connection. That +is, only one thread can be processing a given Proton connection (and +all of its child elements, such as links and deliveries) at a time. +The router honors this requirement by restricting access to a given +Proton connection (and its children) to a single worker thread at a +time. To say this another way, a particular Proton connection can be +processed by any worker threads but not concurrently. + +A worker thread is driven by the Proton proactor API. The worker's +main loop blocks on the proactor waiting for events, processes +incoming events, then blocks again. + + +Core Thread +=== + +The one core thread has several responsibilities, including: + + *) Managing the forwarding state + *) Forwarding messages across the router + *) Forwarding disposition and settlement state changes across the router + *) Managing link credit flow + *) Running the management Agent + +The core thread can be thought of as sitting in between the worker +threads, moving messages and state between them. + +When a worker thread needs to forward a received message it passes the +message and associated delivery state to the core thread. The core +thread uses address information from the message to determine the +outgoing link(s) for the message. The core then queues the message to +the proper outgoing link(s). The core wakes the worker thread(s) (via +the Proton proactor) so the message(s) can be written out the Proton +link. + +When delivery disposition or settlement changes are detected by a +worker thread it notifies the core thread. The core thread then +finds the link or delivery to which the changes need to be propagated Review comment: "finds the link or delivery" suggests a search. It's important to note that the core thread maintains linkages between incoming and outgoing deliveries and incoming and outgoing routed-links. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ted-ross commented on a change in pull request #560: NO-JIRA: add a document explaining the router's threading implementat…
ted-ross commented on a change in pull request #560: NO-JIRA: add a document explaining the router's threading implementat… URL: https://github.com/apache/qpid-dispatch/pull/560#discussion_r367985067 ## File path: docs/notes/threading-info.txt ## @@ -0,0 +1,563 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + + += +Router Threading and Interprocess Communication Guide += + +The Qpid Dispatch Router (qdrouterd) threading architecture is based +on two classes of threads: + + *) A Worker thread which interfaces with the Proton subsystem, and + *) The Core Routing thread which manages the routing database and +performs forwarding. + +In a running router there is a single Core Routing thread (core) and +one or more Worker threads. The number of Worker threads is +determined by the router configuration setting "workerThreads", which +defaults to four. + +IPC between these threads is done using Action queues, Work queues, +and manipulation of shared data. + + +Worker Threads +== + +The Worker thread is responsible for interfacing with the Proton +subsystem. Only a worker thread can safely call directly into the +Proton library. The core thread must communicate with a worker thread +in order to have the worker manipulate Proton state on the core's +behalf. + +The Proton subsystem limits concurrency to a single connection. That +is, only one thread can be processing a given Proton connection (and +all of its child elements, such as links and deliveries) at a time. +The router honors this requirement by restricting access to a given +Proton connection (and its children) to a single worker thread at a +time. To say this another way, a particular Proton connection can be +processed by any worker threads but not concurrently. + +A worker thread is driven by the Proton proactor API. The worker's +main loop blocks on the proactor waiting for events, processes +incoming events, then blocks again. + + +Core Thread +=== + +The one core thread has several responsibilities, including: + + *) Managing the forwarding state + *) Forwarding messages across the router + *) Forwarding disposition and settlement state changes across the router + *) Managing link credit flow + *) Running the management Agent Review comment: I would rephrase this: Responding to management requests for data owned by the core thread. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
