[jira] [Commented] (DISPATCH-1780) multicast support for http 1.1 adaptor
[
https://issues.apache.org/jira/browse/DISPATCH-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250722#comment-17250722
]
ASF GitHub Bot commented on DISPATCH-1780:
--
kgiusti commented on a change in pull request #961:
URL: https://github.com/apache/qpid-dispatch/pull/961#discussion_r544606515
##
File path: src/adaptors/http1/http1_codec.c
##
@@ -181,6 +184,8 @@ struct h1_codec_connection_t {
bool is_request;
bool is_chunked;
+char *boundary_marker;//used for multipart content
Review comment:
Should the boundary_marker be freed+reset in the ecoder_reset() function?
##
File path: src/python_embedded.c
##
@@ -907,3 +907,72 @@ void qd_python_unlock(qd_python_lock_state_t lock_state)
lock_held = false;
sys_mutex_unlock(ilock);
}
+
+void qd_json_msgs_init(PyObject **msgs)
+{
+qd_python_lock_state_t lock_state = qd_python_lock();
+*msgs = PyList_New(0);
+qd_python_unlock(lock_state);
+}
+
+void qd_json_msgs_done(PyObject *msgs)
+{
+qd_python_lock_state_t lock_state = qd_python_lock();
+Py_DECREF(msgs);
+qd_python_unlock(lock_state);
+}
+
+void qd_json_msgs_append(PyObject *msgs, qd_message_t *msg)
+{
+//
+// Parse the message through the body and exit if the message is not well
formed.
+//
+if (qd_message_check_depth(msg, QD_DEPTH_BODY) != QD_MESSAGE_DEPTH_OK)
+return;
+
+// This is called from non-python threads so we need to acquire the GIL to
use python APIS.
+qd_python_lock_state_t lock_state = qd_python_lock();
+PyObject *py_msg = PyObject_CallFunction(message_type, NULL);
+if (!py_msg) {
+qd_error_py();
+qd_python_unlock(lock_state);
+return;
+}
+iter_to_py_attr(qd_message_field_iterator(msg, QD_FIELD_CONTENT_TYPE),
py_iter_copy, py_msg, "content_type");
+iter_to_py_attr(qd_message_field_iterator(msg,
QD_FIELD_APPLICATION_PROPERTIES), py_iter_parse, py_msg, "properties");
+iter_to_py_attr(qd_message_field_iterator(msg, QD_FIELD_BODY),
py_iter_parse, py_msg, "body");
+
+PyList_Append(msgs, py_msg);
+
+Py_DECREF(py_msg);
+qd_error_py();
+qd_python_unlock(lock_state);
+}
+
+char *qd_json_msgs_string(PyObject *msgs)
+{
+qd_python_lock_state_t lock_state = qd_python_lock();
+
+PyObject *message_module =
PyImport_ImportModule("qpid_dispatch_internal.router.message");
+if (!message_module) {
+Py_DECREF(message_module);
+qd_python_unlock(lock_state);
+return NULL;
+}
+PyObject *messages_to_json = PyObject_GetAttrString(message_module,
"messages_to_json");
Review comment:
Py_DECREF messages_to_json needed IIUC
##
File path: src/adaptors/http1/http1_server.c
##
@@ -70,6 +70,7 @@ typedef struct _server_request_t {
boolrequest_settled; // set by adaptor
boolrequest_acked; // true if dispo sent to core
boolheaders_encoded; // True when header encode done
+boolresponse_settled;
Review comment:
Sorry - late night but for the life of me I don't see where this flag is
set! I'd assume it would be set in qdr_http1_server_core_delivery_update when
the outcome for the response arrives...
##
File path: src/adaptors/http1/http1_client.c
##
@@ -1238,6 +1505,24 @@ uint64_t
qdr_http1_client_core_link_deliver(qdr_http1_adaptor_t*adaptor,
_client_response_msg_t *rmsg = DEQ_TAIL(hreq->responses);
assert(rmsg && rmsg->dlv == delivery);
+// when aggregating responses, they are saved on the list until
+// the request has been settled, then encoded in the configured
+// aggregation format
+if (hconn->cfg.aggregation != QD_AGGREGATION_NONE) {
+qd_log(qdr_http1_adaptor->log, QD_LOG_DEBUG, "[C%"PRIu64"][L%"PRIu64"]
Received response (%i responses received), settling", hconn->conn_id,
link->identity, DEQ_SIZE(hreq->responses));
+rmsg->dispo = PN_ACCEPTED;
+qd_message_set_send_complete(msg);
Review comment:
Should this be predicated on qd_message_receive_complete(msg)? I assume
responses may be fragmented and we want to avoid updating the outcome until it
is done arriving.
##
File path: src/adaptors/http1/http1_client.c
##
@@ -1238,6 +1505,24 @@ uint64_t
qdr_http1_client_core_link_deliver(qdr_http1_adaptor_t*adaptor,
_client_response_msg_t *rmsg = DEQ_TAIL(hreq->responses);
assert(rmsg && rmsg->dlv == delivery);
+// when aggregating responses, they are saved on the list until
+// the request has been settled, then encoded in the configured
+// aggregation format
+if (hconn->cfg.aggregation != QD_AGGREGATION_NONE) {
+qd_log(qdr_http1_adaptor->log, QD_LOG_DEBUG, "[C%"PRIu64"][L%"PRIu64"]
Received response (%i responses received), settling", hconn->conn_id,
[GitHub] [qpid-dispatch] kgiusti commented on a change in pull request #961: DISPATCH-1780: initial support for aggregated multicast
kgiusti commented on a change in pull request #961:
URL: https://github.com/apache/qpid-dispatch/pull/961#discussion_r544606515
##
File path: src/adaptors/http1/http1_codec.c
##
@@ -181,6 +184,8 @@ struct h1_codec_connection_t {
bool is_request;
bool is_chunked;
+char *boundary_marker;//used for multipart content
Review comment:
Should the boundary_marker be freed+reset in the ecoder_reset() function?
##
File path: src/python_embedded.c
##
@@ -907,3 +907,72 @@ void qd_python_unlock(qd_python_lock_state_t lock_state)
lock_held = false;
sys_mutex_unlock(ilock);
}
+
+void qd_json_msgs_init(PyObject **msgs)
+{
+qd_python_lock_state_t lock_state = qd_python_lock();
+*msgs = PyList_New(0);
+qd_python_unlock(lock_state);
+}
+
+void qd_json_msgs_done(PyObject *msgs)
+{
+qd_python_lock_state_t lock_state = qd_python_lock();
+Py_DECREF(msgs);
+qd_python_unlock(lock_state);
+}
+
+void qd_json_msgs_append(PyObject *msgs, qd_message_t *msg)
+{
+//
+// Parse the message through the body and exit if the message is not well
formed.
+//
+if (qd_message_check_depth(msg, QD_DEPTH_BODY) != QD_MESSAGE_DEPTH_OK)
+return;
+
+// This is called from non-python threads so we need to acquire the GIL to
use python APIS.
+qd_python_lock_state_t lock_state = qd_python_lock();
+PyObject *py_msg = PyObject_CallFunction(message_type, NULL);
+if (!py_msg) {
+qd_error_py();
+qd_python_unlock(lock_state);
+return;
+}
+iter_to_py_attr(qd_message_field_iterator(msg, QD_FIELD_CONTENT_TYPE),
py_iter_copy, py_msg, "content_type");
+iter_to_py_attr(qd_message_field_iterator(msg,
QD_FIELD_APPLICATION_PROPERTIES), py_iter_parse, py_msg, "properties");
+iter_to_py_attr(qd_message_field_iterator(msg, QD_FIELD_BODY),
py_iter_parse, py_msg, "body");
+
+PyList_Append(msgs, py_msg);
+
+Py_DECREF(py_msg);
+qd_error_py();
+qd_python_unlock(lock_state);
+}
+
+char *qd_json_msgs_string(PyObject *msgs)
+{
+qd_python_lock_state_t lock_state = qd_python_lock();
+
+PyObject *message_module =
PyImport_ImportModule("qpid_dispatch_internal.router.message");
+if (!message_module) {
+Py_DECREF(message_module);
+qd_python_unlock(lock_state);
+return NULL;
+}
+PyObject *messages_to_json = PyObject_GetAttrString(message_module,
"messages_to_json");
Review comment:
Py_DECREF messages_to_json needed IIUC
##
File path: src/adaptors/http1/http1_server.c
##
@@ -70,6 +70,7 @@ typedef struct _server_request_t {
boolrequest_settled; // set by adaptor
boolrequest_acked; // true if dispo sent to core
boolheaders_encoded; // True when header encode done
+boolresponse_settled;
Review comment:
Sorry - late night but for the life of me I don't see where this flag is
set! I'd assume it would be set in qdr_http1_server_core_delivery_update when
the outcome for the response arrives...
##
File path: src/adaptors/http1/http1_client.c
##
@@ -1238,6 +1505,24 @@ uint64_t
qdr_http1_client_core_link_deliver(qdr_http1_adaptor_t*adaptor,
_client_response_msg_t *rmsg = DEQ_TAIL(hreq->responses);
assert(rmsg && rmsg->dlv == delivery);
+// when aggregating responses, they are saved on the list until
+// the request has been settled, then encoded in the configured
+// aggregation format
+if (hconn->cfg.aggregation != QD_AGGREGATION_NONE) {
+qd_log(qdr_http1_adaptor->log, QD_LOG_DEBUG, "[C%"PRIu64"][L%"PRIu64"]
Received response (%i responses received), settling", hconn->conn_id,
link->identity, DEQ_SIZE(hreq->responses));
+rmsg->dispo = PN_ACCEPTED;
+qd_message_set_send_complete(msg);
Review comment:
Should this be predicated on qd_message_receive_complete(msg)? I assume
responses may be fragmented and we want to avoid updating the outcome until it
is done arriving.
##
File path: src/adaptors/http1/http1_client.c
##
@@ -1238,6 +1505,24 @@ uint64_t
qdr_http1_client_core_link_deliver(qdr_http1_adaptor_t*adaptor,
_client_response_msg_t *rmsg = DEQ_TAIL(hreq->responses);
assert(rmsg && rmsg->dlv == delivery);
+// when aggregating responses, they are saved on the list until
+// the request has been settled, then encoded in the configured
+// aggregation format
+if (hconn->cfg.aggregation != QD_AGGREGATION_NONE) {
+qd_log(qdr_http1_adaptor->log, QD_LOG_DEBUG, "[C%"PRIu64"][L%"PRIu64"]
Received response (%i responses received), settling", hconn->conn_id,
link->identity, DEQ_SIZE(hreq->responses));
+rmsg->dispo = PN_ACCEPTED;
+qd_message_set_send_complete(msg);
+qdr_link_flow(qdr_http1_adaptor->core, link, 1, false);
+qdr_delivery_remote_state_updated(qdr_http1_adaptor->core,
[jira] [Commented] (DISPATCH-1887) Add ability toverride host header in http connector
[ https://issues.apache.org/jira/browse/DISPATCH-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250661#comment-17250661 ] ASF GitHub Bot commented on DISPATCH-1887: -- grs opened a new pull request #963: URL: https://github.com/apache/qpid-dispatch/pull/963 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add ability toverride host header in http connector > --- > > Key: DISPATCH-1887 > URL: https://issues.apache.org/jira/browse/DISPATCH-1887 > Project: Qpid Dispatch > Issue Type: Improvement > Components: Protocol Adaptors >Reporter: Gordon Sim >Assignee: Gordon Sim >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] grs opened a new pull request #963: DISPATCH-1887: host override option
grs opened a new pull request #963: URL: https://github.com/apache/qpid-dispatch/pull/963 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (DISPATCH-1887) Add ability toverride host header in http connector
Gordon Sim created DISPATCH-1887: Summary: Add ability toverride host header in http connector Key: DISPATCH-1887 URL: https://issues.apache.org/jira/browse/DISPATCH-1887 Project: Qpid Dispatch Issue Type: Improvement Components: Protocol Adaptors Reporter: Gordon Sim Assignee: Gordon Sim -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1854) Delivery id numbers could be added for better log message comprehension
[ https://issues.apache.org/jira/browse/DISPATCH-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250650#comment-17250650 ] ASF GitHub Bot commented on DISPATCH-1854: -- ChugR opened a new pull request #962: URL: https://github.com/apache/qpid-dispatch/pull/962 Delivery Id is similar to connection id and link id. It will be printed with log statements as a common prefix allowing for easier delivery tracking. Implementation notes: 1. Connection/Link/Delivery Id caching This patch saves the connection id in links and the connection and link ids in the delivery. The these copies are made once at object creation and used as necessary. The cached copies eliminate hunting for the values at log-statement time. The caching is slightly complicated by initial delivery handoff in the adaptors. That handoff is logged as the delivery's connection and link ids are rewritten. For instance: TCP_ADAPTOR (debug) [C1][L1][D121] initial_delivery ownership passed to [C22][L68][D121] Here a TCP_ADAPTOR egress dispatcher on [C1][L1] is passing delivery [D121] to tcp [C22][L68]. The cache strategy may print connection and link ids after the connection or link has disappeared. That's usually not a problem and the strategy eliminates the defensive code required to test if the connection or link still exists. 2. Delivery ids replace printing the address of the delivery Delivery addresses get reused a lot and grepping for them is hard. 3. Common macros to print the connection, link, and delivery ids from a delivery DLV_FMT - the format string defining conn-link-delivery log prefix DLV_ARGS - accessor to get log prefix values from delivery DLV_FMT is a quoted string similar to PRIu64 and PRIu32 and as such must be used outside of quotation marks in the source. DLV_ARGS(dlv) takes an argument which is a pointer to a qd_delivery_t object. A typical usage changes code like this: -qd_log(core->log, QD_LOG_DEBUG, "Delivery decref_CT: dlv:%lx rc:%"PRIu32" link:%"PRIu64" %s", - (long) dlv, ref_count - 1, link_identity, label); +qd_log(core->log, QD_LOG_DEBUG, DLV_FMT" Delivery decref_CT: rc:%"PRIu32" %s", + DLV_ARGS(dlv), ref_count - 1, label); This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Delivery id numbers could be added for better log message comprehension > --- > > Key: DISPATCH-1854 > URL: https://issues.apache.org/jira/browse/DISPATCH-1854 > Project: Qpid Dispatch > Issue Type: Improvement > Components: Protocol Adaptors, Router Node, Routing Engine >Affects Versions: 1.14.0 >Reporter: Charles E. Rolke >Assignee: Charles E. Rolke >Priority: Major > > Deliveries could use id numbers similar to qd_server_t.next_connection_id > numbers. > Then the delivery id numbers could be used in places where the qd_delivery_t > object addresses are printed in logs. This would eliminate the ambiguity with > the object addresses that get reused. > The effort to scavenge the extra info for logs could be wrapped in > qd_log_enabled conditionals so that the work is done only when the log is to > be printed. > Where possible log lines with delivery info could be prefixed with > connection, link, and delivery id numbers like *[C12][L34][D105]* > Delivery ids could be added to transfer and settlement log lines. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ChugR opened a new pull request #962: DISPATCH-1854: Add delivery id to be printed in log prefix
ChugR opened a new pull request #962: URL: https://github.com/apache/qpid-dispatch/pull/962 Delivery Id is similar to connection id and link id. It will be printed with log statements as a common prefix allowing for easier delivery tracking. Implementation notes: 1. Connection/Link/Delivery Id caching This patch saves the connection id in links and the connection and link ids in the delivery. The these copies are made once at object creation and used as necessary. The cached copies eliminate hunting for the values at log-statement time. The caching is slightly complicated by initial delivery handoff in the adaptors. That handoff is logged as the delivery's connection and link ids are rewritten. For instance: TCP_ADAPTOR (debug) [C1][L1][D121] initial_delivery ownership passed to [C22][L68][D121] Here a TCP_ADAPTOR egress dispatcher on [C1][L1] is passing delivery [D121] to tcp [C22][L68]. The cache strategy may print connection and link ids after the connection or link has disappeared. That's usually not a problem and the strategy eliminates the defensive code required to test if the connection or link still exists. 2. Delivery ids replace printing the address of the delivery Delivery addresses get reused a lot and grepping for them is hard. 3. Common macros to print the connection, link, and delivery ids from a delivery DLV_FMT - the format string defining conn-link-delivery log prefix DLV_ARGS - accessor to get log prefix values from delivery DLV_FMT is a quoted string similar to PRIu64 and PRIu32 and as such must be used outside of quotation marks in the source. DLV_ARGS(dlv) takes an argument which is a pointer to a qd_delivery_t object. A typical usage changes code like this: -qd_log(core->log, QD_LOG_DEBUG, "Delivery decref_CT: dlv:%lx rc:%"PRIu32" link:%"PRIu64" %s", - (long) dlv, ref_count - 1, link_identity, label); +qd_log(core->log, QD_LOG_DEBUG, DLV_FMT" Delivery decref_CT: rc:%"PRIu32" %s", + DLV_ARGS(dlv), ref_count - 1, label); This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1780) multicast support for http 1.1 adaptor
[ https://issues.apache.org/jira/browse/DISPATCH-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250559#comment-17250559 ] ASF GitHub Bot commented on DISPATCH-1780: -- grs opened a new pull request #961: URL: https://github.com/apache/qpid-dispatch/pull/961 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > multicast support for http 1.1 adaptor > -- > > Key: DISPATCH-1780 > URL: https://issues.apache.org/jira/browse/DISPATCH-1780 > Project: Qpid Dispatch > Issue Type: Improvement >Reporter: Gordon Sim >Assignee: Gordon Sim >Priority: Major > Fix For: 1.15.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] grs opened a new pull request #961: DISPATCH-1780: initial support for aggregated multicast
grs opened a new pull request #961: URL: https://github.com/apache/qpid-dispatch/pull/961 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1886) Review and fix races between connection activation and closure
[ https://issues.apache.org/jira/browse/DISPATCH-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250512#comment-17250512 ] ASF GitHub Bot commented on DISPATCH-1886: -- ted-ross merged pull request #960: URL: https://github.com/apache/qpid-dispatch/pull/960 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Review and fix races between connection activation and closure > -- > > Key: DISPATCH-1886 > URL: https://issues.apache.org/jira/browse/DISPATCH-1886 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Reporter: Ted Ross >Assignee: Ted Ross >Priority: Major > Fix For: 1.15.0 > > > [~cliffjansen] reviewed the usage of connection-wake (activation) in the > router code and identified some potential issues with the multi-threading > that occurs between activation and IO processing. > This Jira will be used to track updates to the code that result from this > analysis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1886) Review and fix races between connection activation and closure
[ https://issues.apache.org/jira/browse/DISPATCH-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250511#comment-17250511 ] ASF subversion and git services commented on DISPATCH-1886: --- Commit d7215f378e3a2f5bd8c01071fc6cd4cc1b5120a2 in qpid-dispatch's branch refs/heads/master from Ted Ross [ https://gitbox.apache.org/repos/asf?p=qpid-dispatch.git;h=d7215f3 ] DISPATCH-1886 - Close race window on connection activate/close. > Review and fix races between connection activation and closure > -- > > Key: DISPATCH-1886 > URL: https://issues.apache.org/jira/browse/DISPATCH-1886 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Reporter: Ted Ross >Assignee: Ted Ross >Priority: Major > Fix For: 1.15.0 > > > [~cliffjansen] reviewed the usage of connection-wake (activation) in the > router code and identified some potential issues with the multi-threading > that occurs between activation and IO processing. > This Jira will be used to track updates to the code that result from this > analysis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ted-ross merged pull request #960: DISPATCH-1886 - Close race window on connection activate/close.
ted-ross merged pull request #960: URL: https://github.com/apache/qpid-dispatch/pull/960 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1886) Review and fix races between connection activation and closure
[ https://issues.apache.org/jira/browse/DISPATCH-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250433#comment-17250433 ] ASF GitHub Bot commented on DISPATCH-1886: -- ganeshmurthy commented on pull request #960: URL: https://github.com/apache/qpid-dispatch/pull/960#issuecomment-746578890 Just a quick note that the server->conn_activation_lock was introduced as a fix to https://issues.apache.org/jira/browse/DISPATCH-1417 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Review and fix races between connection activation and closure > -- > > Key: DISPATCH-1886 > URL: https://issues.apache.org/jira/browse/DISPATCH-1886 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Reporter: Ted Ross >Assignee: Ted Ross >Priority: Major > Fix For: 1.15.0 > > > [~cliffjansen] reviewed the usage of connection-wake (activation) in the > router code and identified some potential issues with the multi-threading > that occurs between activation and IO processing. > This Jira will be used to track updates to the code that result from this > analysis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ganeshmurthy commented on pull request #960: DISPATCH-1886 - Close race window on connection activate/close.
ganeshmurthy commented on pull request #960: URL: https://github.com/apache/qpid-dispatch/pull/960#issuecomment-746578890 Just a quick note that the server->conn_activation_lock was introduced as a fix to https://issues.apache.org/jira/browse/DISPATCH-1417 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1886) Review and fix races between connection activation and closure
[ https://issues.apache.org/jira/browse/DISPATCH-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250430#comment-17250430 ] ASF GitHub Bot commented on DISPATCH-1886: -- ganeshmurthy commented on pull request #960: URL: https://github.com/apache/qpid-dispatch/pull/960#issuecomment-746575563 This looks good to me. Every call to qd_server_activate(conn); must be protected by the server->conn_activation_lock. 1. The call to qd_server_activate in CORE_connection_activate() (in router_node.c) is already protected by this lock. 2. This PR removes qd_link_activate() 3. This PR introduces the lock around qd_server_activate call in qd_connection_invoke_deferred (in server.c) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Review and fix races between connection activation and closure > -- > > Key: DISPATCH-1886 > URL: https://issues.apache.org/jira/browse/DISPATCH-1886 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Reporter: Ted Ross >Assignee: Ted Ross >Priority: Major > Fix For: 1.15.0 > > > [~cliffjansen] reviewed the usage of connection-wake (activation) in the > router code and identified some potential issues with the multi-threading > that occurs between activation and IO processing. > This Jira will be used to track updates to the code that result from this > analysis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ganeshmurthy commented on pull request #960: DISPATCH-1886 - Close race window on connection activate/close.
ganeshmurthy commented on pull request #960: URL: https://github.com/apache/qpid-dispatch/pull/960#issuecomment-746575563 This looks good to me. Every call to qd_server_activate(conn); must be protected by the server->conn_activation_lock. 1. The call to qd_server_activate in CORE_connection_activate() (in router_node.c) is already protected by this lock. 2. This PR removes qd_link_activate() 3. This PR introduces the lock around qd_server_activate call in qd_connection_invoke_deferred (in server.c) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1886) Review and fix races between connection activation and closure
[ https://issues.apache.org/jira/browse/DISPATCH-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250414#comment-17250414 ] ASF GitHub Bot commented on DISPATCH-1886: -- ted-ross opened a new pull request #960: URL: https://github.com/apache/qpid-dispatch/pull/960 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Review and fix races between connection activation and closure > -- > > Key: DISPATCH-1886 > URL: https://issues.apache.org/jira/browse/DISPATCH-1886 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Reporter: Ted Ross >Assignee: Ted Ross >Priority: Major > Fix For: 1.15.0 > > > [~cliffjansen] reviewed the usage of connection-wake (activation) in the > router code and identified some potential issues with the multi-threading > that occurs between activation and IO processing. > This Jira will be used to track updates to the code that result from this > analysis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ted-ross opened a new pull request #960: DISPATCH-1886 - Close race window on connection activate/close.
ted-ross opened a new pull request #960: URL: https://github.com/apache/qpid-dispatch/pull/960 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Resolved] (DISPATCH-1883) [Test] Travis Xenial AMD64 should have Python 'selectors' module
[ https://issues.apache.org/jira/browse/DISPATCH-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles E. Rolke resolved DISPATCH-1883. Fix Version/s: 1.15.0 Resolution: Fixed Fixed at Commit e0fa62 Patch adds python selectors to README, travis, github workflow, and dockerfiles. > [Test] Travis Xenial AMD64 should have Python 'selectors' module > > > Key: DISPATCH-1883 > URL: https://issues.apache.org/jira/browse/DISPATCH-1883 > Project: Qpid Dispatch > Issue Type: Improvement >Reporter: Charles E. Rolke >Priority: Major > Fix For: 1.15.0 > > > https://travis-ci.com/github/apache/qpid-dispatch/builds/208502016 > All tests are skipped because 'selectors' is absent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (DISPATCH-1886) Review and fix races between connection activation and closure
Ted Ross created DISPATCH-1886: -- Summary: Review and fix races between connection activation and closure Key: DISPATCH-1886 URL: https://issues.apache.org/jira/browse/DISPATCH-1886 Project: Qpid Dispatch Issue Type: Bug Components: Container Reporter: Ted Ross Assignee: Ted Ross Fix For: 1.15.0 [~cliffjansen] reviewed the usage of connection-wake (activation) in the router code and identified some potential issues with the multi-threading that occurs between activation and IO processing. This Jira will be used to track updates to the code that result from this analysis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1883) [Test] Travis Xenial AMD64 should have Python 'selectors' module
[ https://issues.apache.org/jira/browse/DISPATCH-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250400#comment-17250400 ] ASF subversion and git services commented on DISPATCH-1883: --- Commit e0fa62c117557207239f603ff77ba68c4e2cee59 in qpid-dispatch's branch refs/heads/master from Charles E. Rolke [ https://gitbox.apache.org/repos/asf?p=qpid-dispatch.git;h=e0fa62c ] DISPATCH-1883: Tcp self test helpers require python selectors * add requirement to README * include selectors in github workflow, travis, and dockerfiles This closes #949 > [Test] Travis Xenial AMD64 should have Python 'selectors' module > > > Key: DISPATCH-1883 > URL: https://issues.apache.org/jira/browse/DISPATCH-1883 > Project: Qpid Dispatch > Issue Type: Improvement >Reporter: Charles E. Rolke >Priority: Major > > https://travis-ci.com/github/apache/qpid-dispatch/builds/208502016 > All tests are skipped because 'selectors' is absent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1883) [Test] Travis Xenial AMD64 should have Python 'selectors' module
[ https://issues.apache.org/jira/browse/DISPATCH-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250401#comment-17250401 ] ASF GitHub Bot commented on DISPATCH-1883: -- asfgit closed pull request #949: URL: https://github.com/apache/qpid-dispatch/pull/949 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > [Test] Travis Xenial AMD64 should have Python 'selectors' module > > > Key: DISPATCH-1883 > URL: https://issues.apache.org/jira/browse/DISPATCH-1883 > Project: Qpid Dispatch > Issue Type: Improvement >Reporter: Charles E. Rolke >Priority: Major > > https://travis-ci.com/github/apache/qpid-dispatch/builds/208502016 > All tests are skipped because 'selectors' is absent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] asfgit closed pull request #949: DISPATCH-1883: Install selectors to run tcp echo server/client tools
asfgit closed pull request #949: URL: https://github.com/apache/qpid-dispatch/pull/949 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Resolved] (DISPATCH-1884) TCP Adaptor fails asan leak tests
[ https://issues.apache.org/jira/browse/DISPATCH-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Ross resolved DISPATCH-1884. Resolution: Fixed > TCP Adaptor fails asan leak tests > - > > Key: DISPATCH-1884 > URL: https://issues.apache.org/jira/browse/DISPATCH-1884 > Project: Qpid Dispatch > Issue Type: Bug > Components: Protocol Adaptors >Reporter: Ted Ross >Assignee: Ted Ross >Priority: Minor > Fix For: 1.15.0 > > > There are a number of shutdown leaks reported by asan during the TCP adaptor > tests. Some of these leaks are real in that connectors and listeners that > are dynamically created and destroyed will leak parts of their configuration > and state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1883) [Test] Travis Xenial AMD64 should have Python 'selectors' module
[ https://issues.apache.org/jira/browse/DISPATCH-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250357#comment-17250357 ] ASF GitHub Bot commented on DISPATCH-1883: -- ChugR commented on a change in pull request #949: URL: https://github.com/apache/qpid-dispatch/pull/949#discussion_r544355560 ## File path: .travis.yml ## @@ -69,6 +69,8 @@ jobs: - python -m pip install --user --upgrade tox virtualenv==20.0.23 # Install quart to run the http2 tests. - python -m pip install --user quart +# DISPATCH-1883: Install selectors to run tcp echo server/client tools +- python -m pip install --user selectors Review comment: yes. well spotted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > [Test] Travis Xenial AMD64 should have Python 'selectors' module > > > Key: DISPATCH-1883 > URL: https://issues.apache.org/jira/browse/DISPATCH-1883 > Project: Qpid Dispatch > Issue Type: Improvement >Reporter: Charles E. Rolke >Priority: Major > > https://travis-ci.com/github/apache/qpid-dispatch/builds/208502016 > All tests are skipped because 'selectors' is absent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ChugR commented on a change in pull request #949: DISPATCH-1883: Install selectors to run tcp echo server/client tools
ChugR commented on a change in pull request #949: URL: https://github.com/apache/qpid-dispatch/pull/949#discussion_r544355560 ## File path: .travis.yml ## @@ -69,6 +69,8 @@ jobs: - python -m pip install --user --upgrade tox virtualenv==20.0.23 # Install quart to run the http2 tests. - python -m pip install --user quart +# DISPATCH-1883: Install selectors to run tcp echo server/client tools +- python -m pip install --user selectors Review comment: yes. well spotted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-8489) Connection thread looping
[
https://issues.apache.org/jira/browse/QPID-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250226#comment-17250226
]
Daniil Kirilyuk commented on QPID-8489:
---
Hi [~orudyy],
Thank you very much for the links to the JDK issues, and sorry for the delay -
it took some time for testing.
With the latest JDK 11 problem could not be reproduced indeed:
{noformat}
java --version
openjdk 11.0.9 2020-10-20
OpenJDK Runtime Environment 18.9 (build 11.0.9+11)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.9+11, mixed mode)
{noformat}
Log under JDK 11.0.9:
{noformat}
2020-12-16 10:27:25,570 INFO [IO-/172.23.38.36:41814]
(o.a.q.s.t.NonBlockingConnectionTLSDelegate) - isOutboundDone(): false,
isInboundDone(): true, _status: Status = OK HandshakeStatus = NOT_HANDSHAKING
bytesConsumed = 0 bytesProduced = 0
2020-12-16 10:27:38,946 INFO [IO-/172.23.38.36:41814]
(o.a.q.s.t.NonBlockingConnectionTLSDelegate) - isOutboundDone(): false,
isInboundDone(): true, _status: Status = OK HandshakeStatus = NOT_HANDSHAKING
bytesConsumed = 0 bytesProduced = 0
2020-12-16 10:27:38,946 INFO [IO-/172.23.38.36:41814]
(o.a.q.s.t.NonBlockingConnectionTLSDelegate) - isOutboundDone(): false,
isInboundDone(): true, _status: Status = OK HandshakeStatus = NOT_HANDSHAKING
bytesConsumed = 0 bytesProduced = 0
2020-12-16 10:27:38,949 INFO [Broker-Config] (q.m.c.dropped_connection) -
[con:5(TWBXT_TWEB2FOAADEVCLR@null/default)] CON-1004 : Connection dropped
{noformat}
Unfortunately we have to provide support for Java 8 (as well as the TLSv1.3).
As for JDK-8214418 some details could be found in the [mailing
list|http://mail.openjdk.java.net/pipermail/security-dev/2019-January/019142.html]
and [here|http://cr.openjdk.java.net/~xuelei/8214418/webrev.00/open.patch].
Regarding the fix you suggested earlier: could the condition be made more
strict, e.g.:
{noformat}
if (_status.bytesProduced() < 1 && _status.getHandshakeStatus() ==
SSLEngineResult.HandshakeStatus.NEED_WRAP && !_sslEngine.isOutboundDone() &&
_sslEngine.isInboundDone())
{
throw new SSLException(String.format("SSLEngine.wrap produced 0 bytes
(status %s, handshake status %s)", _status.getStatus(),
_status.getHandshakeStatus()));
}
{noformat}
This check seems to handle correctly both the tight looping initiation as well
as the normal connection lifecycle.
> Connection thread looping
> -
>
> Key: QPID-8489
> URL: https://issues.apache.org/jira/browse/QPID-8489
> Project: Qpid
> Issue Type: Bug
> Components: Broker-J
>Affects Versions: qpid-java-broker-8.0.2
>Reporter: Daniil Kirilyuk
>Priority: Minor
> Attachments:
> 0001-QPID-8489-Break-an-infinite-connection-processing-lo.patch, QPID-8489 -
> java.ssl.debug.log, QPID-8489.log, simple_recv.cpp, thread-dump.st
>
>
> Error happens quite rarely and is not easy to reproduce. Although the problem
> was partly fixed by fixing QPID-8477, it still can be reproduced. The main
> symptom is significant increase of CPU usage even when no messages are sent
> to broker anymore. CPU usage can rise from 30% to 90% and higher, making
> broker unusable. After such CPU rise the only way to fix broker will be
> restarting it.
> Analysis has shown, that error occurs with CPP proton client in cases when
> 1) SSL connection is used
> 2) connection errors on client side are ignored
> 3) connection is dropped due to the client process termination / network
> disconnection
> *Steps to reproduce*
> # Java broker should be installed
> # Broker should be configured to allow one connection
> # Prepare certificates
> # Install Qpid::Proton 0.28.0
> wget
> [http://archive.apache.org/dist/qpid/proton/0.28.0/qpid-proton-0.28.0.tar.gz]
> gunzip qpid-proton-0.28.0.tar.gz
> mkdir -p qpid-proton-0.28.0/build && pushd qpid-proton-0.28.0/build && cmake
> .. && make all && popd
> # Replace and edit example *qpid-proton-0.28.0/cpp/examples/simple_recv.cpp*
> with the one attached
> # Build again
> cd qpid-proton-0.28.0/build
> make
> # Break the broker
> ./cpp/examples/simple_recv & ./cpp/examples/simple_recv
> Connection error
> {color:#ffab00}^C <= Hit Ctrl+C to kill process{color}
> # {color:#172b4d}If CPU usage didn't increased, find the PID of the first
> simple_recv process using ps-ef | grep simple_recv and kill it using kill -9
> PID.{color}
> *Analysis*
> CPU usage rises when connection is dropped on the client side or when network
> is broken between client and broker. The main point is that client isn't well
> behaved and connection shouldn't be closed correctly.
> On broker side connection becomes "orphaned": it is still maintained by
> broker, but no real reading / writing is performed. Following method calls
> are performed in an endless loop for each "orphaned" connection:
> SelectorThread.performSelect()
>
[jira] [Updated] (PROTON-2307) Allow access to connection properties in cpp binding
[ https://issues.apache.org/jira/browse/PROTON-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pete Fawcett updated PROTON-2307: - Labels: enhancement (was: enhancement pull-request-available) > Allow access to connection properties in cpp binding > > > Key: PROTON-2307 > URL: https://issues.apache.org/jira/browse/PROTON-2307 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: proton-c-0.33.0 >Reporter: Pete Fawcett >Priority: Minor > Labels: enhancement > > I would like to be able to set the "custom" properties on a connection using > the C++ bindings. > The initial reason for this is to allow a client program to set the > "qpid.client_process" property on a connection to a broker as this is useful > when identifying the connection on the QMF console. > I have written code to enable this and hope to submit a pull request soon. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
