[jira] [Created] (PROTON-2001) inbuilt external mechanism ignores initial response
Gordon Sim created PROTON-2001: -- Summary: inbuilt external mechanism ignores initial response Key: PROTON-2001 URL: https://issues.apache.org/jira/browse/PROTON-2001 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: proton-c-0.26.0 Reporter: Gordon Sim The SASL EXTERNAL mechanism allows the initial response to be used to pass the identity the client wishes to be authorized as. The inbuilt proton implementation of external ignores this unlike cyrus sasl. Not a huge issue I think, but noting here in case it is useful. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] k-wall opened a new pull request #30: PROTON-1998: Add SASL protocol trace
k-wall opened a new pull request #30: PROTON-1998: Add SASL protocol trace URL: https://github.com/apache/qpid-proton-j/pull/30 This patch adds a SASL protocol trace that is controlled by the same mechanism as the main protocol trace. Could you comment on the approach? If it is acceptable, I'll add some tests. One unsettled question in my mind ProtocolTracer: at the moment, ProtocolTracer implementations don't receive SaslFrameBodies. I did as this would break ProtocolTracer impls (including QpidJMS's - which would ClassCastException). I wondered using a default method ProtocolTracer#ignoreSaslFrameBody which would return true. Comments appreciated.. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1998) [Proton-J] Add SASL protocol trace
[
https://issues.apache.org/jira/browse/PROTON-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757192#comment-16757192
]
ASF GitHub Bot commented on PROTON-1998:
k-wall commented on pull request #30: PROTON-1998: Add SASL protocol trace
URL: https://github.com/apache/qpid-proton-j/pull/30
This patch adds a SASL protocol trace that is controlled by the same
mechanism as the main protocol trace.
Could you comment on the approach? If it is acceptable, I'll add some tests.
One unsettled question in my mind ProtocolTracer: at the moment,
ProtocolTracer implementations don't receive SaslFrameBodies. I did as this
would break ProtocolTracer impls (including QpidJMS's - which would
ClassCastException). I wondered using a default method
ProtocolTracer#ignoreSaslFrameBody which would return true. Comments
appreciated..
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [Proton-J] Add SASL protocol trace
> --
>
> Key: PROTON-1998
> URL: https://issues.apache.org/jira/browse/PROTON-1998
> Project: Qpid Proton
> Issue Type: Improvement
> Components: proton-j
>Reporter: Keith Wall
>Priority: Minor
>
> Unlike Proton, Proton-J does not provide SASL frame trace if environment
> variable PN_TRACE_FRM is set. It would be useful if Proton-J had this
> ability too to help diagnose SASL negotiation problem.
> Proton's SASL frame trace looks like this:
> {code:java}
> [0x7fc112c03a00]: -> SASL
> [0x7fc112c03a00]: <- SASL
> [0x7fc112c03a00]:0 <- @sasl-mechanisms(64)
> [sasl-server-mechanisms=@PN_SYMBOL[:ANONYMOUS]]
> [0x7fc112c03a00]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS,
> initial-response=b"guest@Oslo.local"]
> [0x7fc112c03a00]:0 <- @sasl-outcome(68) [code=0]
> [0x7fc112c03a00]: -> AMQP
> [0x7fc112c03a00]:0 -> @open(16)
> [container-id="be777c26-af6e-4935-a6be-316cc8bbdb35", hostname="127.0.0.1",
> channel-max=32767]{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] k-wall commented on issue #30: PROTON-1998: Add SASL protocol trace
k-wall commented on issue #30: PROTON-1998: Add SASL protocol trace URL: https://github.com/apache/qpid-proton-j/pull/30#issuecomment-459340846 I answered my own question regarding the ProtocolTracer in my second commit. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1998) [Proton-J] Add SASL protocol trace
[
https://issues.apache.org/jira/browse/PROTON-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757229#comment-16757229
]
ASF GitHub Bot commented on PROTON-1998:
k-wall commented on issue #30: PROTON-1998: Add SASL protocol trace
URL: https://github.com/apache/qpid-proton-j/pull/30#issuecomment-459340846
I answered my own question regarding the ProtocolTracer in my second commit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [Proton-J] Add SASL protocol trace
> --
>
> Key: PROTON-1998
> URL: https://issues.apache.org/jira/browse/PROTON-1998
> Project: Qpid Proton
> Issue Type: Improvement
> Components: proton-j
>Reporter: Keith Wall
>Priority: Minor
>
> Unlike Proton, Proton-J does not provide SASL frame trace if environment
> variable PN_TRACE_FRM is set. It would be useful if Proton-J had this
> ability too to help diagnose SASL negotiation problem.
> Proton's SASL frame trace looks like this:
> {code:java}
> [0x7fc112c03a00]: -> SASL
> [0x7fc112c03a00]: <- SASL
> [0x7fc112c03a00]:0 <- @sasl-mechanisms(64)
> [sasl-server-mechanisms=@PN_SYMBOL[:ANONYMOUS]]
> [0x7fc112c03a00]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS,
> initial-response=b"guest@Oslo.local"]
> [0x7fc112c03a00]:0 <- @sasl-outcome(68) [code=0]
> [0x7fc112c03a00]: -> AMQP
> [0x7fc112c03a00]:0 -> @open(16)
> [container-id="be777c26-af6e-4935-a6be-316cc8bbdb35", hostname="127.0.0.1",
> channel-max=32767]{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1246) Address Proxy module leaks qdrc_endpoint_t on shutdown
[
https://issues.apache.org/jira/browse/DISPATCH-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757281#comment-16757281
]
ASF GitHub Bot commented on DISPATCH-1246:
--
ChugR commented on pull request #445: DISPATCH-1246: clean up core link
endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252687262
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
This branch produces a valgrind InvalidRead when the work item is used after
it is free'd.
`
kind = InvalidRead (count=1)
Invalid read of size 1
Stack:
(qdr_deliver_continue_peers_CT)
/home/chug/git/qpid-dispatch/src/router_core/transfer.c:1236
(qdr_deliver_continue_CT)
/home/chug/git/qpid-dispatch/src/router_core/transfer.c:1269
`
But it doesn't look like the work was free'd by this code:
`
Address 0x143aaa79 is 41 bytes inside a block of size 48 free'd:
(free)
/builddir/build/BUILD/valgrind-3.14.0/coregrind/m_replacemalloc/vg_replace_malloc.c:540
(free_qdr_link_work_t)
/home/chug/git/qpid-dispatch/src/router_core/router_core.c:36
(qdr_connection_process)
/home/chug/git/qpid-dispatch/src/router_core/connections.c:341
`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Address Proxy module leaks qdrc_endpoint_t on shutdown
> --
>
> Key: DISPATCH-1246
> URL: https://issues.apache.org/jira/browse/DISPATCH-1246
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Routing Engine
>Affects Versions: 1.5.0
>Reporter: Ken Giusti
>Assignee: Chuck Rolke
>Priority: Minor
> Fix For: 1.6.0
>
>
> Running the system_tests_edge_router under valgrind:
> ==21590== 657 (256 direct, 401 indirect) bytes in 4 blocks are definitely
> lost in loss record 2,886 of 3,532
> ==21590==at 0x4C3147C: memalign (vg_replace_malloc.c:908)
> ==21590==by 0x4C31589: posix_memalign (vg_replace_malloc.c:1072)
> ==21590==by 0x4EA12FF: qd_alloc (alloc_pool.c:181)
> ==21590==by 0x4E83086: qdrc_endpoint_create_link_CT
> (core_link_endpoint.c:70)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:237)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:199)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E8123F: qdr_connection_opened_CT (connections.c:1147)
> ==21590==by 0x4E8C916: router_core_thread (router_core_thread.c:124)
> ==21590==by 0x5511593: start_thread (in /usr/lib64/libpthread-2.27.so)
> ==21590==by 0x62A6F4E: clone (in /usr/lib64/libc-2.27.so)
> ==21590==
> Appears that the endpoint allocated here in addr_proxy.c:
> //
> // Attach a receiving link for edge address tracking updates.
> //
> ap->tracking_endpoint =
> qdrc_endpoint_create_link_CT(ap->core, conn, QD_INCOMING,
>
> qdr_terminus_normal(QD_TERMINUS_EDGE_ADDRESS_TRACKING),
> qdr_terminus(0),
> &ap->endpoint_descriptor, ap);
> is not released when the router is shutdown
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] ChugR commented on a change in pull request #445: DISPATCH-1246: clean up core link endpoints on shutdown
ChugR commented on a change in pull request #445: DISPATCH-1246: clean up core
link endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252687262
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
This branch produces a valgrind InvalidRead when the work item is used after
it is free'd.
`
kind = InvalidRead (count=1)
Invalid read of size 1
Stack:
(qdr_deliver_continue_peers_CT)
/home/chug/git/qpid-dispatch/src/router_core/transfer.c:1236
(qdr_deliver_continue_CT)
/home/chug/git/qpid-dispatch/src/router_core/transfer.c:1269
`
But it doesn't look like the work was free'd by this code:
`
Address 0x143aaa79 is 41 bytes inside a block of size 48 free'd:
(free)
/builddir/build/BUILD/valgrind-3.14.0/coregrind/m_replacemalloc/vg_replace_malloc.c:540
(free_qdr_link_work_t)
/home/chug/git/qpid-dispatch/src/router_core/router_core.c:36
(qdr_connection_process)
/home/chug/git/qpid-dispatch/src/router_core/connections.c:341
`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] kgiusti commented on a change in pull request #445: DISPATCH-1246: clean up core link endpoints on shutdown
kgiusti commented on a change in pull request #445: DISPATCH-1246: clean up
core link endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252694243
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
Good catch Chuck!
Did this happen running the unit tests? I'd like to reproduce it.
-1 this PR until I can fix this
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1246) Address Proxy module leaks qdrc_endpoint_t on shutdown
[
https://issues.apache.org/jira/browse/DISPATCH-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757304#comment-16757304
]
ASF GitHub Bot commented on DISPATCH-1246:
--
kgiusti commented on pull request #445: DISPATCH-1246: clean up core link
endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252694243
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
Good catch Chuck!
Did this happen running the unit tests? I'd like to reproduce it.
-1 this PR until I can fix this
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Address Proxy module leaks qdrc_endpoint_t on shutdown
> --
>
> Key: DISPATCH-1246
> URL: https://issues.apache.org/jira/browse/DISPATCH-1246
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Routing Engine
>Affects Versions: 1.5.0
>Reporter: Ken Giusti
>Assignee: Chuck Rolke
>Priority: Minor
> Fix For: 1.6.0
>
>
> Running the system_tests_edge_router under valgrind:
> ==21590== 657 (256 direct, 401 indirect) bytes in 4 blocks are definitely
> lost in loss record 2,886 of 3,532
> ==21590==at 0x4C3147C: memalign (vg_replace_malloc.c:908)
> ==21590==by 0x4C31589: posix_memalign (vg_replace_malloc.c:1072)
> ==21590==by 0x4EA12FF: qd_alloc (alloc_pool.c:181)
> ==21590==by 0x4E83086: qdrc_endpoint_create_link_CT
> (core_link_endpoint.c:70)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:237)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:199)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E8123F: qdr_connection_opened_CT (connections.c:1147)
> ==21590==by 0x4E8C916: router_core_thread (router_core_thread.c:124)
> ==21590==by 0x5511593: start_thread (in /usr/lib64/libpthread-2.27.so)
> ==21590==by 0x62A6F4E: clone (in /usr/lib64/libc-2.27.so)
> ==21590==
> Appears that the endpoint allocated here in addr_proxy.c:
> //
> // Attach a receiving link for edge address tracking updates.
> //
> ap->tracking_endpoint =
> qdrc_endpoint_create_link_CT(ap->core, conn, QD_INCOMING,
>
> qdr_terminus_normal(QD_TERMINUS_EDGE_ADDRESS_TRACKING),
> qdr_terminus(0),
> &ap->endpoint_descriptor, ap);
> is not released when the router is shutdown
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1999) [c] Crash in pn_connection_finalize
[
https://issues.apache.org/jira/browse/PROTON-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757315#comment-16757315
]
Olivier Delbeke commented on PROTON-1999:
-
The proposed solution seems to fix the issue. I am no longer able to reproduce
it, and it's OK for me to close the ticket (as "not a bug").
However, I still don't really understand why it solves the problem. The
proton::sender::worker() (or actually proton::link:work_queue()) is just a
const getter function, so I don't see how calling this could harm anything or
return a different value than what it would return if called from the right
thread.
> [c] Crash in pn_connection_finalize
> ---
>
> Key: PROTON-1999
> URL: https://issues.apache.org/jira/browse/PROTON-1999
> Project: Qpid Proton
> Issue Type: Bug
> Components: cpp-binding, proton-c
>Affects Versions: proton-c-0.26.0
> Environment: Linux 64-bits (Ubuntu 16.04 and Oracle Linux 7.4)
>Reporter: Olivier Delbeke
>Assignee: Cliff Jansen
>Priority: Major
> Attachments: call_stack.txt, example2.cpp, log.txt, main.cpp,
> run_qpid-broker.sh
>
>
> Here is my situation : I have several proton::containers (~20).
> Each one has its own proton::messaging_handler, and handles one
> proton::connection to a local qpid-broker (everything runs on the same Linux
> machine).
> 20 x ( one container with one handler with one connection with one link)
> Some containers/connections/handlers work in send mode ; they have one link
> that is a proton::sender.
> Some containers/connections/handlers work in receive mode ; they have one
> link that is a proton::receiver. Each time they receive an input message,
> they do some processing on it, and finally add a "sender->send()" task to the
> work queue of some sender handlers ( by calling work_queue()->add( [=] \{
> sender->send(msg); } as shown in the multi-threading examples).
> This works fine for some time (tens of thousands of messages, several minutes
> or hours), but eventually crashes, either with a SEGFAULT (when the
> qpid-proton lib is compiled in release mode) or with an assert (in debug
> mode), in qpid-proton/c/src/core/engine.c line 483,
> assert(!conn->transport->referenced) in function pn_connection_finalize().
> The proton logs (activated with export PN_TRACE_FRM=1) do not show anything
> abnormal (no loss of connection, no rejection of messages, no timeouts, ...).
> As the connection is not closed, I wonder why pn_connection_finalize() would
> be called in the first place.
> I joined the logs and the call trace.
> Happens on 0.26.0 but also reproduced with the latest master (Jan 28, 2019).
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (QPID-8276) [Broker-J] Broker can leak closed NonBlockingConnection objects and eventually run out of heap memory
Alex Rudyy created QPID-8276:
Summary: [Broker-J] Broker can leak closed NonBlockingConnection
objects and eventually run out of heap memory
Key: QPID-8276
URL: https://issues.apache.org/jira/browse/QPID-8276
Project: Qpid
Issue Type: Improvement
Components: Broker-J
Affects Versions: qpid-java-broker-7.0.6, qpid-java-broker-7.0.5,
qpid-java-broker-7.0.4, qpid-java-broker-7.1.0, qpid-java-6.1.7,
qpid-java-broker-7.0.1, qpid-java-broker-7.0.0, qpid-java-broker-7.0.2,
qpid-java-broker-7.0.3
Reporter: Alex Rudyy
Fix For: qpid-java-broker-7.0.7, qpid-java-broker-7.1.1
The Qpid Broker-J can leak closed NonBlockingConnection objects.
The heap dump analysis of impacted broker instance revealed that leaked
{{NonBlockingConnection}} objects are accumulated in
{{SelectorThread.SelectionTask#_unscheduledConnections}} belonging to AMQP port
IO pool. They have no ticker set and no state changed flag set
({{NonBlockingConnection#isStateChanged() == false)}}. As result, the
NonBlockingConnection objects are not removed from
{{SelectorThread#_unscheduledConnections}} on invocation of
{{SelectorThread.SelectionTask#processUnscheduledConnections()}} called from
{{SelectorThread.SelectionTask#performSelect()}}.
The {{NonBlockingConnection}} and underlying model object are in closed state.
It seems that leaked {{NonBlockingConnection}} was closed as part of
invocation {{NonBlockingConnection#doWork()}}. The connection was unregistered
on {{VirtualHost}} IO pool and re-registered with port IO pool as part of
invocation {{NetworkConnectionScheduler#processConnection}} At first, it was
stored in collection {{SelectorThread.SelectionTask#_unregisteredConnections}}.
Later on, it was moved from
{{SelectorThread.SelectionTask#_unregisteredConnections}} to
{{SelectorThread.SelectionTask#_unscheduledConnections}} as part of invocation
{{SelectorThread.SelectionTask#reregisterUnregisteredConnections}} and stack
there afterwards.
The TLS transport was used in leaked connection, but, I think that connection
with plain transport can be leaked as well.
I suspect that connections were leaked in result of following scenario:
* Invocation of {{SocketChannel#read(java.nio.ByteBuffer[])}} returned {{-1}}
in {{NonBlockingConnection#readFromNetwork}}.
* The flag {{NonBlockingConnection#_closed}} was set to {{true}}. The method
{{ProtocolEngine#notifyWork()}} was not invoked to set {{state changed}} flag
to {{true}}
* The execution of {{NonBlockingConnection#doWork()}} ended up it connection
shutdown (due to {{NonBlockingConnection#_closed}} being set) and following
re-scheduling the connection on port IO scheduler. The latter resulted in
connection being put into
{{SelectorThread.SelectionTask#_unscheduledConnections}} as described above.
It seems that opening and closing frequent connections with connection life
span {{>10s}} (required for tickers to be removed) can ended-up in connections
being leaked as described in scenario above. It looks like connections which
are closed orderly or closed in result of {{IOException}} being thrown from
socket read/write operation are not effected by the defect.
The impacted broker instance can eventually crash with out of memory error.
Broker memory monitoring and periodic broker restarts can mitigate the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-8276) [Broker-J] Broker can leak closed NonBlockingConnection objects and eventually run out of heap memory
[
https://issues.apache.org/jira/browse/QPID-8276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rudyy updated QPID-8276:
-
Issue Type: Bug (was: Improvement)
> [Broker-J] Broker can leak closed NonBlockingConnection objects and
> eventually run out of heap memory
> -
>
> Key: QPID-8276
> URL: https://issues.apache.org/jira/browse/QPID-8276
> Project: Qpid
> Issue Type: Bug
> Components: Broker-J
>Affects Versions: qpid-java-broker-7.0.3, qpid-java-broker-7.0.2,
> qpid-java-broker-7.0.0, qpid-java-broker-7.0.1, qpid-java-6.1.7,
> qpid-java-broker-7.1.0, qpid-java-broker-7.0.4, qpid-java-broker-7.0.5,
> qpid-java-broker-7.0.6
>Reporter: Alex Rudyy
>Priority: Critical
> Fix For: qpid-java-broker-7.0.7, qpid-java-broker-7.1.1
>
>
> The Qpid Broker-J can leak closed NonBlockingConnection objects.
> The heap dump analysis of impacted broker instance revealed that leaked
> {{NonBlockingConnection}} objects are accumulated in
> {{SelectorThread.SelectionTask#_unscheduledConnections}} belonging to AMQP
> port IO pool. They have no ticker set and no state changed flag set
> ({{NonBlockingConnection#isStateChanged() == false)}}. As result, the
> NonBlockingConnection objects are not removed from
> {{SelectorThread#_unscheduledConnections}} on invocation of
> {{SelectorThread.SelectionTask#processUnscheduledConnections()}} called from
> {{SelectorThread.SelectionTask#performSelect()}}.
> The {{NonBlockingConnection}} and underlying model object are in closed state.
> It seems that leaked {{NonBlockingConnection}} was closed as part of
> invocation {{NonBlockingConnection#doWork()}}. The connection was
> unregistered on {{VirtualHost}} IO pool and re-registered with port IO pool
> as part of invocation {{NetworkConnectionScheduler#processConnection}} At
> first, it was stored in collection
> {{SelectorThread.SelectionTask#_unregisteredConnections}}. Later on, it was
> moved from {{SelectorThread.SelectionTask#_unregisteredConnections}} to
> {{SelectorThread.SelectionTask#_unscheduledConnections}} as part of
> invocation {{SelectorThread.SelectionTask#reregisterUnregisteredConnections}}
> and stack there afterwards.
> The TLS transport was used in leaked connection, but, I think that connection
> with plain transport can be leaked as well.
> I suspect that connections were leaked in result of following scenario:
> * Invocation of {{SocketChannel#read(java.nio.ByteBuffer[])}} returned
> {{-1}} in {{NonBlockingConnection#readFromNetwork}}.
> * The flag {{NonBlockingConnection#_closed}} was set to {{true}}. The method
> {{ProtocolEngine#notifyWork()}} was not invoked to set {{state changed}} flag
> to {{true}}
> * The execution of {{NonBlockingConnection#doWork()}} ended up it connection
> shutdown (due to {{NonBlockingConnection#_closed}} being set) and following
> re-scheduling the connection on port IO scheduler. The latter resulted in
> connection being put into
> {{SelectorThread.SelectionTask#_unscheduledConnections}} as described above.
> It seems that opening and closing frequent connections with connection life
> span {{>10s}} (required for tickers to be removed) can ended-up in
> connections being leaked as described in scenario above. It looks like
> connections which are closed orderly or closed in result of {{IOException}}
> being thrown from socket read/write operation are not effected by the defect.
> The impacted broker instance can eventually crash with out of memory error.
> Broker memory monitoring and periodic broker restarts can mitigate the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1246) Address Proxy module leaks qdrc_endpoint_t on shutdown
[
https://issues.apache.org/jira/browse/DISPATCH-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757475#comment-16757475
]
ASF GitHub Bot commented on DISPATCH-1246:
--
ChugR commented on pull request #445: DISPATCH-1246: clean up core link
endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252745594
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
I'm running a Debug build. This error does not happen on every run. Also,
I'm seeing an occasional failure with an assert link_route_proxy.c#371, a lrp
pointer points to a proxy with bad proxy_state.
Attached are two files that have the grinder result from two runs only of
system_tests_edge_router.
- grinder-at-1246.txt has the bad memory reference and the assert.
- grinder-at-1246-2.txt has no bad reference, no assert, and a varying set
of object leaks.
[grinder-at-1246.txt](https://github.com/apache/qpid-dispatch/files/2818175/grinder-at-1246.txt)
[grinder-at-1246-2.txt](https://github.com/apache/qpid-dispatch/files/2818176/grinder-at-1246-2.txt)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Address Proxy module leaks qdrc_endpoint_t on shutdown
> --
>
> Key: DISPATCH-1246
> URL: https://issues.apache.org/jira/browse/DISPATCH-1246
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Routing Engine
>Affects Versions: 1.5.0
>Reporter: Ken Giusti
>Assignee: Chuck Rolke
>Priority: Minor
> Fix For: 1.6.0
>
>
> Running the system_tests_edge_router under valgrind:
> ==21590== 657 (256 direct, 401 indirect) bytes in 4 blocks are definitely
> lost in loss record 2,886 of 3,532
> ==21590==at 0x4C3147C: memalign (vg_replace_malloc.c:908)
> ==21590==by 0x4C31589: posix_memalign (vg_replace_malloc.c:1072)
> ==21590==by 0x4EA12FF: qd_alloc (alloc_pool.c:181)
> ==21590==by 0x4E83086: qdrc_endpoint_create_link_CT
> (core_link_endpoint.c:70)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:237)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:199)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E8123F: qdr_connection_opened_CT (connections.c:1147)
> ==21590==by 0x4E8C916: router_core_thread (router_core_thread.c:124)
> ==21590==by 0x5511593: start_thread (in /usr/lib64/libpthread-2.27.so)
> ==21590==by 0x62A6F4E: clone (in /usr/lib64/libc-2.27.so)
> ==21590==
> Appears that the endpoint allocated here in addr_proxy.c:
> //
> // Attach a receiving link for edge address tracking updates.
> //
> ap->tracking_endpoint =
> qdrc_endpoint_create_link_CT(ap->core, conn, QD_INCOMING,
>
> qdr_terminus_normal(QD_TERMINUS_EDGE_ADDRESS_TRACKING),
> qdr_terminus(0),
> &ap->endpoint_descriptor, ap);
> is not released when the router is shutdown
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] ChugR commented on a change in pull request #445: DISPATCH-1246: clean up core link endpoints on shutdown
ChugR commented on a change in pull request #445: DISPATCH-1246: clean up core
link endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252745594
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
I'm running a Debug build. This error does not happen on every run. Also,
I'm seeing an occasional failure with an assert link_route_proxy.c#371, a lrp
pointer points to a proxy with bad proxy_state.
Attached are two files that have the grinder result from two runs only of
system_tests_edge_router.
- grinder-at-1246.txt has the bad memory reference and the assert.
- grinder-at-1246-2.txt has no bad reference, no assert, and a varying set
of object leaks.
[grinder-at-1246.txt](https://github.com/apache/qpid-dispatch/files/2818175/grinder-at-1246.txt)
[grinder-at-1246-2.txt](https://github.com/apache/qpid-dispatch/files/2818176/grinder-at-1246-2.txt)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1246) Address Proxy module leaks qdrc_endpoint_t on shutdown
[
https://issues.apache.org/jira/browse/DISPATCH-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757502#comment-16757502
]
ASF GitHub Bot commented on DISPATCH-1246:
--
kgiusti commented on pull request #445: DISPATCH-1246: clean up core link
endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252750736
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
I have a theory - more like a wild*ss guess - on why that abort() call may
be in error. Testing now.
That use after free is scary: a link work item is being accessed by the core
thread after being freed by an I/O thread. I wonder if the peer link's work
should only be accessed under the conn_lock?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Address Proxy module leaks qdrc_endpoint_t on shutdown
> --
>
> Key: DISPATCH-1246
> URL: https://issues.apache.org/jira/browse/DISPATCH-1246
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Routing Engine
>Affects Versions: 1.5.0
>Reporter: Ken Giusti
>Assignee: Chuck Rolke
>Priority: Minor
> Fix For: 1.6.0
>
>
> Running the system_tests_edge_router under valgrind:
> ==21590== 657 (256 direct, 401 indirect) bytes in 4 blocks are definitely
> lost in loss record 2,886 of 3,532
> ==21590==at 0x4C3147C: memalign (vg_replace_malloc.c:908)
> ==21590==by 0x4C31589: posix_memalign (vg_replace_malloc.c:1072)
> ==21590==by 0x4EA12FF: qd_alloc (alloc_pool.c:181)
> ==21590==by 0x4E83086: qdrc_endpoint_create_link_CT
> (core_link_endpoint.c:70)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:237)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:199)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E8123F: qdr_connection_opened_CT (connections.c:1147)
> ==21590==by 0x4E8C916: router_core_thread (router_core_thread.c:124)
> ==21590==by 0x5511593: start_thread (in /usr/lib64/libpthread-2.27.so)
> ==21590==by 0x62A6F4E: clone (in /usr/lib64/libc-2.27.so)
> ==21590==
> Appears that the endpoint allocated here in addr_proxy.c:
> //
> // Attach a receiving link for edge address tracking updates.
> //
> ap->tracking_endpoint =
> qdrc_endpoint_create_link_CT(ap->core, conn, QD_INCOMING,
>
> qdr_terminus_normal(QD_TERMINUS_EDGE_ADDRESS_TRACKING),
> qdr_terminus(0),
> &ap->endpoint_descriptor, ap);
> is not released when the router is shutdown
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] kgiusti commented on a change in pull request #445: DISPATCH-1246: clean up core link endpoints on shutdown
kgiusti commented on a change in pull request #445: DISPATCH-1246: clean up
core link endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252750736
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
I have a theory - more like a wild*ss guess - on why that abort() call may
be in error. Testing now.
That use after free is scary: a link work item is being accessed by the core
thread after being freed by an I/O thread. I wonder if the peer link's work
should only be accessed under the conn_lock?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Assigned] (PROTON-1997) [Python] Urls class is not exported by reactor
[ https://issues.apache.org/jira/browse/PROTON-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Ross reassigned PROTON-1997: --- Assignee: Justin Ross (was: Andrew Stitcher) > [Python] Urls class is not exported by reactor > -- > > Key: PROTON-1997 > URL: https://issues.apache.org/jira/browse/PROTON-1997 > Project: Qpid Proton > Issue Type: Bug > Components: python-binding >Affects Versions: proton-c-0.26.0 >Reporter: Radim Kubis >Assignee: Justin Ross >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] kgiusti commented on a change in pull request #445: DISPATCH-1246: clean up core link endpoints on shutdown
kgiusti commented on a change in pull request #445: DISPATCH-1246: clean up
core link endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252826040
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
@ChugR - I've removed the abort() call. It's possible to get a second
delete event when the connection is dropped with outstanding deletes pending.
Running the system_tests_edge_router testcase under valgrind kept hanging
for me. I traced this to a test that wasn't cleaning up properly (running the
router under valgrind slowed down the test to the point where the bug was
evident). I've pushed the fix to master, you may want to rebase on that.
https://github.com/apache/qpid-dispatch/commit/d10676f8ee6a76fff57f6a62cfa4d583dabc63a9
Other than that I do see leaks for other objects - mostly deliveries - but
no link endpoint leaks.
And I haven't hit the bad memory reference. This appears to be a race, but
it may not be a result of this patch per se. I'm looking a bit at the locking
around the link work lists in any case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1246) Address Proxy module leaks qdrc_endpoint_t on shutdown
[
https://issues.apache.org/jira/browse/DISPATCH-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757699#comment-16757699
]
ASF GitHub Bot commented on DISPATCH-1246:
--
kgiusti commented on pull request #445: DISPATCH-1246: clean up core link
endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252826040
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
@ChugR - I've removed the abort() call. It's possible to get a second
delete event when the connection is dropped with outstanding deletes pending.
Running the system_tests_edge_router testcase under valgrind kept hanging
for me. I traced this to a test that wasn't cleaning up properly (running the
router under valgrind slowed down the test to the point where the bug was
evident). I've pushed the fix to master, you may want to rebase on that.
https://github.com/apache/qpid-dispatch/commit/d10676f8ee6a76fff57f6a62cfa4d583dabc63a9
Other than that I do see leaks for other objects - mostly deliveries - but
no link endpoint leaks.
And I haven't hit the bad memory reference. This appears to be a race, but
it may not be a result of this patch per se. I'm looking a bit at the locking
around the link work lists in any case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Address Proxy module leaks qdrc_endpoint_t on shutdown
> --
>
> Key: DISPATCH-1246
> URL: https://issues.apache.org/jira/browse/DISPATCH-1246
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Routing Engine
>Affects Versions: 1.5.0
>Reporter: Ken Giusti
>Assignee: Chuck Rolke
>Priority: Minor
> Fix For: 1.6.0
>
>
> Running the system_tests_edge_router under valgrind:
> ==21590== 657 (256 direct, 401 indirect) bytes in 4 blocks are definitely
> lost in loss record 2,886 of 3,532
> ==21590==at 0x4C3147C: memalign (vg_replace_malloc.c:908)
> ==21590==by 0x4C31589: posix_memalign (vg_replace_malloc.c:1072)
> ==21590==by 0x4EA12FF: qd_alloc (alloc_pool.c:181)
> ==21590==by 0x4E83086: qdrc_endpoint_create_link_CT
> (core_link_endpoint.c:70)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:237)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:199)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E8123F: qdr_connection_opened_CT (connections.c:1147)
> ==21590==by 0x4E8C916: router_core_thread (router_core_thread.c:124)
> ==21590==by 0x5511593: start_thread (in /usr/lib64/libpthread-2.27.so)
> ==21590==by 0x62A6F4E: clone (in /usr/lib64/libc-2.27.so)
> ==21590==
> Appears that the endpoint allocated here in addr_proxy.c:
> //
> // Attach a receiving link for edge address tracking updates.
> //
> ap->tracking_endpoint =
> qdrc_endpoint_create_link_CT(ap->core, conn, QD_INCOMING,
>
> qdr_terminus_normal(QD_TERMINUS_EDGE_ADDRESS_TRACKING),
> qdr_terminus(0),
> &ap->endpoint_descriptor, ap);
> is not released when the router is shutdown
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] ChugR commented on a change in pull request #445: DISPATCH-1246: clean up core link endpoints on shutdown
ChugR commented on a change in pull request #445: DISPATCH-1246: clean up core
link endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252844491
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
Great improvement. I've limited my tests to running
system_tests_edge_router. I see leaks of **qdr_delivery_t**, **qd_link_ref_t**,
and **qdr_link_work_t** only over repeated runs.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (DISPATCH-1259) delivery->link_work race condition
Ken Giusti created DISPATCH-1259:
Summary: delivery->link_work race condition
Key: DISPATCH-1259
URL: https://issues.apache.org/jira/browse/DISPATCH-1259
Project: Qpid Dispatch
Issue Type: Bug
Components: Router Node
Affects Versions: 1.5.0
Reporter: Ken Giusti
Assignee: Ken Giusti
Fix For: 1.6.0
[~chug] hit the following use-after-free error under valgrind:
{code:java}
kind = InvalidRead (count=1)
Invalid read of size 1
Stack:
(qdr_deliver_continue_peers_CT)
/home/chug/git/qpid-dispatch/src/router_core/transfer.c:1236
(qdr_deliver_continue_CT)
/home/chug/git/qpid-dispatch/src/router_core/transfer.c:1269
(router_core_thread)
/home/chug/git/qpid-dispatch/src/router_core/router_core_thread.c:148
(start_thread)
/usr/src/debug/glibc-2.28-60-g4d7af7815a/nptl/pthread_create.c:486
(clone)
/usr/src/debug/glibc-2.28-60-g4d7af7815a/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Address 0x143aaa79 is 41 bytes inside a block of size 48 free'd:
(free)
/builddir/build/BUILD/valgrind-3.14.0/coregrind/m_replacemalloc/vg_replace_malloc.c:540
(free_qdr_link_work_t)
/home/chug/git/qpid-dispatch/src/router_core/router_core.c:36
(qdr_connection_process)
/home/chug/git/qpid-dispatch/src/router_core/connections.c:341
(AMQP_writable_conn_handler)
/home/chug/git/qpid-dispatch/src/router_node.c:174
(writable_handler) /home/chug/git/qpid-dispatch/src/container.c:332
(qd_container_handle_event) /home/chug/git/qpid-dispatch/src/container.c:640
(handle) /home/chug/git/qpid-dispatch/src/server.c:985
(thread_run) /home/chug/git/qpid-dispatch/src/server.c:1010
(qd_server_run) /home/chug/git/qpid-dispatch/src/server.c:1284
(main_process) /home/chug/git/qpid-dispatch/router/src/main.c:112
(main) /home/chug/git/qpid-dispatch/router/src/main.c:367
Block was alloc'd at:
(malloc)
/builddir/build/BUILD/valgrind-3.14.0/coregrind/m_replacemalloc/vg_replace_malloc.c:309
(new_qdr_link_work_t)
/home/chug/git/qpid-dispatch/src/router_core/router_core.c:36
(qdr_forward_deliver_CT)
/home/chug/git/qpid-dispatch/src/router_core/forwarder.c:226
(qdr_forward_multicast_CT)
/home/chug/git/qpid-dispatch/src/router_core/forwarder.c:474
(qdr_forward_message_CT)
/home/chug/git/qpid-dispatch/src/router_core/forwarder.c:995
(qdr_link_forward_CT)
/home/chug/git/qpid-dispatch/src/router_core/transfer.c:918
(qdr_link_deliver_CT)
/home/chug/git/qpid-dispatch/src/router_core/transfer.c:1094
(router_core_thread)
/home/chug/git/qpid-dispatch/src/router_core/router_core_thread.c:148
(start_thread)
/usr/src/debug/glibc-2.28-60-g4d7af7815a/nptl/pthread_create.c:486
(clone)
/usr/src/debug/glibc-2.28-60-g4d7af7815a/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{code}
The router core thread is accessing a link_work object after it was deleted by
the I/O thread.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1246) Address Proxy module leaks qdrc_endpoint_t on shutdown
[
https://issues.apache.org/jira/browse/DISPATCH-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757730#comment-16757730
]
ASF GitHub Bot commented on DISPATCH-1246:
--
ChugR commented on pull request #445: DISPATCH-1246: clean up core link
endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252844491
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
Great improvement. I've limited my tests to running
system_tests_edge_router. I see leaks of **qdr_delivery_t**, **qd_link_ref_t**,
and **qdr_link_work_t** only over repeated runs.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Address Proxy module leaks qdrc_endpoint_t on shutdown
> --
>
> Key: DISPATCH-1246
> URL: https://issues.apache.org/jira/browse/DISPATCH-1246
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Routing Engine
>Affects Versions: 1.5.0
>Reporter: Ken Giusti
>Assignee: Chuck Rolke
>Priority: Minor
> Fix For: 1.6.0
>
>
> Running the system_tests_edge_router under valgrind:
> ==21590== 657 (256 direct, 401 indirect) bytes in 4 blocks are definitely
> lost in loss record 2,886 of 3,532
> ==21590==at 0x4C3147C: memalign (vg_replace_malloc.c:908)
> ==21590==by 0x4C31589: posix_memalign (vg_replace_malloc.c:1072)
> ==21590==by 0x4EA12FF: qd_alloc (alloc_pool.c:181)
> ==21590==by 0x4E83086: qdrc_endpoint_create_link_CT
> (core_link_endpoint.c:70)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:237)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:199)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E8123F: qdr_connection_opened_CT (connections.c:1147)
> ==21590==by 0x4E8C916: router_core_thread (router_core_thread.c:124)
> ==21590==by 0x5511593: start_thread (in /usr/lib64/libpthread-2.27.so)
> ==21590==by 0x62A6F4E: clone (in /usr/lib64/libc-2.27.so)
> ==21590==
> Appears that the endpoint allocated here in addr_proxy.c:
> //
> // Attach a receiving link for edge address tracking updates.
> //
> ap->tracking_endpoint =
> qdrc_endpoint_create_link_CT(ap->core, conn, QD_INCOMING,
>
> qdr_terminus_normal(QD_TERMINUS_EDGE_ADDRESS_TRACKING),
> qdr_terminus(0),
> &ap->endpoint_descriptor, ap);
> is not released when the router is shutdown
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] kgiusti commented on a change in pull request #445: DISPATCH-1246: clean up core link endpoints on shutdown
kgiusti commented on a change in pull request #445: DISPATCH-1246: clean up
core link endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252848035
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
I've opened a separate JIRA for the race -
https://issues.apache.org/jira/browse/DISPATCH-1259
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1246) Address Proxy module leaks qdrc_endpoint_t on shutdown
[
https://issues.apache.org/jira/browse/DISPATCH-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757734#comment-16757734
]
ASF GitHub Bot commented on DISPATCH-1246:
--
kgiusti commented on pull request #445: DISPATCH-1246: clean up core link
endpoints on shutdown
URL: https://github.com/apache/qpid-dispatch/pull/445#discussion_r252848035
##
File path: src/router_core/connections.c
##
@@ -1227,9 +1223,7 @@ static void qdr_connection_closed_CT(qdr_core_t *core,
qdr_action_t *action, boo
qdr_connection_work_t *work = DEQ_HEAD(conn->work_list);
while (work) {
DEQ_REMOVE_HEAD(conn->work_list);
-qdr_terminus_free(work->source);
-qdr_terminus_free(work->target);
-free_qdr_connection_work_t(work);
+qdr_connection_work_free_CT(work);
Review comment:
I've opened a separate JIRA for the race -
https://issues.apache.org/jira/browse/DISPATCH-1259
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Address Proxy module leaks qdrc_endpoint_t on shutdown
> --
>
> Key: DISPATCH-1246
> URL: https://issues.apache.org/jira/browse/DISPATCH-1246
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Routing Engine
>Affects Versions: 1.5.0
>Reporter: Ken Giusti
>Assignee: Chuck Rolke
>Priority: Minor
> Fix For: 1.6.0
>
>
> Running the system_tests_edge_router under valgrind:
> ==21590== 657 (256 direct, 401 indirect) bytes in 4 blocks are definitely
> lost in loss record 2,886 of 3,532
> ==21590==at 0x4C3147C: memalign (vg_replace_malloc.c:908)
> ==21590==by 0x4C31589: posix_memalign (vg_replace_malloc.c:1072)
> ==21590==by 0x4EA12FF: qd_alloc (alloc_pool.c:181)
> ==21590==by 0x4E83086: qdrc_endpoint_create_link_CT
> (core_link_endpoint.c:70)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:237)
> ==21590==by 0x4E92C2C: on_conn_event (addr_proxy.c:199)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E82E28: qdrc_event_conn_raise (core_events.c:90)
> ==21590==by 0x4E8123F: qdr_connection_opened_CT (connections.c:1147)
> ==21590==by 0x4E8C916: router_core_thread (router_core_thread.c:124)
> ==21590==by 0x5511593: start_thread (in /usr/lib64/libpthread-2.27.so)
> ==21590==by 0x62A6F4E: clone (in /usr/lib64/libc-2.27.so)
> ==21590==
> Appears that the endpoint allocated here in addr_proxy.c:
> //
> // Attach a receiving link for edge address tracking updates.
> //
> ap->tracking_endpoint =
> qdrc_endpoint_create_link_CT(ap->core, conn, QD_INCOMING,
>
> qdr_terminus_normal(QD_TERMINUS_EDGE_ADDRESS_TRACKING),
> qdr_terminus(0),
> &ap->endpoint_descriptor, ap);
> is not released when the router is shutdown
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
