Re: Temporary queues deletion
YES!!!
Now temporary queues are deletes as expected!
Huge thanks for help!
Do you think the fix should be included in proton mainline?
Just in case the patch is attached.
Best regards,
On 01.08.2018 14:03, Gordon Sim wrote:
> On 01/08/18 11:44, Michael Ivanov wrote:
>> Currently I have the following:
>>
>> [0x55da51103a80]: -> AMQP
>> [0x55da51103a80]:0 -> @open(16)
>> [container-id="03F84325-E282-4535-800D-637EB4383871", hostname="localhost",
>> channel-max=32767]
>> [0x55da51103a80]:0 -> @begin(17) [next-outgoing-id=0,
>> incoming-window=2147483647, outgoing-window=2147483647]
>> [0x55da51103a80]:0 -> @attach(18) [name="receiver-29591.0", handle=0,
>> role=true, snd-settle-mode=2, rcv-settle-mode=0,
>> source=@source(40) [durable=0, expiry-policy=:"link-detach", timeout=1000,
>> dynamic=true,
>> dynamic-node-properties={:"lifetime-policy"="delete-on-close",
>> :exclusive=true}], target=@target(41) [durable=0,
>> expiry-policy=:"link-detach", timeout=1000, dynamic=false],
>> initial-delivery-count=0, max-message-size=0]
>
> That looks like you are setting the value to the string 'delete-on-close'. It
> should instead by a described list, e.g. something
> like:
>
> static pn_bytes_t _symbol = { 25, "amqp:delete-on-close:list" };
> pn_data_put_described(data);
> pn_data_enter(data);
> pn_data_put_symbol(data, _symbol);
> pn_data_put_list(data);
> pn_data_exit(data);
>
> (Alternatively, you can try patching your broker with the attached patch that
> I think should fix the handling of the exclusive
> option.)
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
--
\ / | |
(OvO) | Mikhail Iwanow |
(^^^) | Voice: +7 (911) 223-1300 |
\^/ | E-mail: iv...@logit-ag.de |
^ ^ | |
diff -ur qpid-proton-0.24.0.orig/c/src/messenger/messenger.c qpid-proton-0.24.0/c/src/messenger/messenger.c
--- qpid-proton-0.24.0.orig/c/src/messenger/messenger.c 2018-06-26 16:42:03.0 +0300
+++ qpid-proton-0.24.0/c/src/messenger/messenger.c 2018-08-01 14:27:05.918510331 +0300
@@ -1763,11 +1766,39 @@
pn_link_set_rcv_settle_mode(link, messenger->rcv_settle_mode);
}
if (pn_streq(name, "#")) {
+pn_data_t *properties;
+
if (pn_link_is_sender(link)) {
pn_terminus_set_dynamic(pn_link_target(link), true);
+ properties = pn_terminus_properties(pn_link_target(link));
} else {
pn_terminus_set_dynamic(pn_link_source(link), true);
+ properties = pn_terminus_properties(pn_link_source(link));
+}
+pn_data_rewind(properties);
+pn_data_next(properties);
+if (pn_data_type(properties) == PN_EOS)
+ pn_data_put_map(properties);
+if (pn_data_type(properties) == PN_MAP) {
+ static pn_bytes_t _exclusive = { 9, "exclusive" };
+ static pn_bytes_t _lifetime_policy = { 15, "lifetime-policy" };
+ static pn_bytes_t _delete_on_close = { 25, "amqp:delete-on-close:list" };
+
+ pn_data_enter(properties);
+ pn_data_put_symbol(properties, _lifetime_policy);
+ pn_data_put_described(properties);
+ pn_data_enter(properties);
+pn_data_put_symbol(properties, _delete_on_close);
+pn_data_put_list(properties);
+ pn_data_exit(properties);
+ pn_data_put_symbol(properties, _exclusive);
+ pn_data_put_bool(properties, true);
+ pn_data_exit(properties);
}
+else
+ pn_logf("%s: unexpected TERMINUS PROPERTIES type [%d], MAP expected",
+__func__, pn_data_type(properties));
+pn_data_rewind(properties);
} else {
pn_terminus_set_address(pn_link_target(link), name);
pn_terminus_set_address(pn_link_source(link), name);
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
On 01/08/18 11:44, Michael Ivanov wrote:
Currently I have the following:
[0x55da51103a80]: -> AMQP
[0x55da51103a80]:0 -> @open(16) [container-id="03F84325-E282-4535-800D-637EB4383871",
hostname="localhost", channel-max=32767]
[0x55da51103a80]:0 -> @begin(17) [next-outgoing-id=0,
incoming-window=2147483647, outgoing-window=2147483647]
[0x55da51103a80]:0 -> @attach(18) [name="receiver-29591.0", handle=0,
role=true, snd-settle-mode=2, rcv-settle-mode=0,
source=@source(40) [durable=0, expiry-policy=:"link-detach", timeout=1000,
dynamic=true,
dynamic-node-properties={:"lifetime-policy"="delete-on-close",
:exclusive=true}], target=@target(41) [durable=0,
expiry-policy=:"link-detach", timeout=1000, dynamic=false],
initial-delivery-count=0, max-message-size=0]
That looks like you are setting the value to the string
'delete-on-close'. It should instead by a described list, e.g. something
like:
static pn_bytes_t _symbol = { 25, "amqp:delete-on-close:list" };
pn_data_put_described(data);
pn_data_enter(data);
pn_data_put_symbol(data, _symbol);
pn_data_put_list(data);
pn_data_exit(data);
(Alternatively, you can try patching your broker with the attached patch
that I think should fix the handling of the exclusive option.)
diff --git a/src/qpid/broker/amqp/NodeProperties.cpp b/src/qpid/broker/amqp/NodeProperties.cpp
index 500a0c0..3fdd986 100644
--- a/src/qpid/broker/amqp/NodeProperties.cpp
+++ b/src/qpid/broker/amqp/NodeProperties.cpp
@@ -377,7 +377,7 @@ std::string NodeProperties::getAlternateExchange() const
bool NodeProperties::trackControllingLink() const
{
-return lifetime == QueueSettings::DELETE_ON_CLOSE || lifetime == QueueSettings::DELETE_IF_EMPTY;
+return exclusive || lifetime == QueueSettings::DELETE_ON_CLOSE || lifetime == QueueSettings::DELETE_IF_EMPTY;
}
const qpid::types::Variant::Map& NodeProperties::getProperties() const
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
Currently I have the following:
[0x55da51103a80]: -> AMQP
[0x55da51103a80]:0 -> @open(16)
[container-id="03F84325-E282-4535-800D-637EB4383871", hostname="localhost",
channel-max=32767]
[0x55da51103a80]:0 -> @begin(17) [next-outgoing-id=0,
incoming-window=2147483647, outgoing-window=2147483647]
[0x55da51103a80]:0 -> @attach(18) [name="receiver-29591.0", handle=0,
role=true, snd-settle-mode=2, rcv-settle-mode=0,
source=@source(40) [durable=0, expiry-policy=:"link-detach", timeout=1000,
dynamic=true,
dynamic-node-properties={:"lifetime-policy"="delete-on-close",
:exclusive=true}], target=@target(41) [durable=0,
expiry-policy=:"link-detach", timeout=1000, dynamic=false],
initial-delivery-count=0, max-message-size=0]
[0x55da51103a80]: <- AMQP
[0x55da51103a80]:0 <- @open(16)
[container-id="c8c172cf-5753-4169-9939-05f63eb56dc5", channel-max=32767,
offered-capabilities=@PN_SYMBOL[:"ANONYMOUS-RELAY"],
properties={:product="qpid-cpp", :version="1.38.0", :platform="Linux",
:host="island"}]
[0x55da51103a80]:0 <- @begin(17) [remote-channel=0, next-outgoing-id=0,
incoming-window=2147483647, outgoing-window=2147483647]
[0x55da51103a80]:0 <- @attach(18) [name="receiver-29591.0", handle=0,
role=false, snd-settle-mode=2, rcv-settle-mode=0,
source=@source(40)
[address="03F84325-E282-4535-800D-637EB4383871_receiver-29591.0", durable=0,
timeout=0, dynamic=false,
dynamic-node-properties={:"supported-dist-modes"="move",
:"lifetime-policy"=@:"amqp:delete-on-no-links:list" [],
:exclusive=true}, distribution-mode=:move], target=@target(41) [durable=0,
timeout=0, dynamic=false], initial-delivery-count=0,
max-message-size=0]
delete-on-close is replaced by delete-on-no-links in response for some reason.
I'm using qpidd 1.38
Best regards,
On 01.08.2018 13:21, Gordon Sim wrote:
> On 31/07/18 21:21, Michael Ivanov wrote:
>> Then I modified messenger.c (starting from line 1768) and set 'exclusive'
>> property there on source terminus when creating queue using '#' specifier
> [...]
>> And after that I have seen that my temporary queues started to be created
>> in exclusive mode, at least qpid-stat shows this:
>>
>> queue dur autoDel excl msg
>> msgIn msgOut bytes bytesIn bytesOut cons bind
>>
>> ==
>> 21948F78- . . . -70568712B_receiver-27663.0 Y Y 1
>> 1 0 435 435 0 1 1
>>
>> But after clien program that has created this queue termionated, the queue
>> regrettably
>> was not deleted, but "exclusive" property was dropped instead:
>>
>> queue dur autoDel excl msg
>> msgIn msgOut bytes bytesIn bytesOut cons bind
>>
>> ==
>> 21948F78- . . . -70568712B_receiver-27663.0 Y 0
>> 1 1 0 435 435 0 1
>>
>> Setting lifetime-property to delete-on-close in same way also does not make
>> any difference.
>>
>> Any hope to get temporary queue deleted without terminating server
>> (responder)
>> process?
>
> How are you setting the lifetime-policy? It needs to be encoded as a
> described value. E.g. the trace would be something like this:
>
>> @attach(18)
>> [name="97e94a05-1e69-492f-b9b4-282beb79484e#_aafc09ac-efc1-4f51-b038-c529408b7b61",
>> handle=0, role=true,
>> snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0,
>> timeout=0, dynamic=true,
>> dynamic-node-properties={:"lifetime-policy"=@:"amqp:delete-on-close:list"
>> []}], target=@target(41) [durable=0, timeout=0,
>> dynamic=false], initial-delivery-count=0, max-message-size=0]
>
> When I do that (I did this using a different client), then the broker will
> indeed delete that queue when the receiver exits. The
> sender is still open at that point, but it gets a detach informing it that
> the queue is deleted.
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
--
\ / | |
(OvO) | Mikhail Iwanow |
(^^^) | Voice: +7 (911) 223-1300 |
\^/ | E-mail: iv...@logit-ag.de |
^ ^ | |
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
On 31/07/18 21:21, Michael Ivanov wrote:
Then I modified messenger.c (starting from line 1768) and set 'exclusive'
property there on source terminus when creating queue using '#' specifier
[...]
And after that I have seen that my temporary queues started to be created
in exclusive mode, at least qpid-stat shows this:
queuedur autoDel excl msg msgIn
msgOut bytes bytesIn bytesOut cons bind
==
21948F78- . . . -70568712B_receiver-27663.0 YY1 1
0 4354350 1 1
But after clien program that has created this queue termionated, the queue
regrettably
was not deleted, but "exclusive" property was dropped instead:
queuedur autoDel excl msg msgIn
msgOut bytes bytesIn bytesOut cons bind
==
21948F78- . . . -70568712B_receiver-27663.0 Y 0 1
1 0435 435 0 1
Setting lifetime-property to delete-on-close in same way also does not make
any difference.
Any hope to get temporary queue deleted without terminating server (responder)
process?
How are you setting the lifetime-policy? It needs to be encoded as a
described value. E.g. the trace would be something like this:
@attach(18) [name="97e94a05-1e69-492f-b9b4-282beb79484e#_aafc09ac-efc1-4f51-b038-c529408b7b61",
handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0, timeout=0,
dynamic=true, dynamic-node-properties={:"lifetime-policy"=@:"amqp:delete-on-close:list"
[]}], target=@target(41) [durable=0, timeout=0, dynamic=false], initial-delivery-count=0, max-message-size=0]
When I do that (I did this using a different client), then the broker
will indeed delete that queue when the receiver exits. The sender is
still open at that point, but it gets a detach informing it that the
queue is deleted.
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
Greetings,
I tried setting lifetime-policy in qmf request and I do not see any difference.
Then I modified messenger.c (starting from line 1768) and set 'exclusive'
property there on source terminus when creating queue using '#' specifier
as follows:
if (pn_streq(name, "#")) {
pn_data_t *properties;
if (pn_link_is_sender(link)) {
pn_terminus_set_dynamic(pn_link_target(link), true);
properties = pn_terminus_properties(pn_link_target(link));
} else {
pn_terminus_set_dynamic(pn_link_source(link), true);
properties = pn_terminus_properties(pn_link_source(link));
}
pn_data_rewind(properties);
pn_data_next(properties);
if (pn_data_type(properties) == PN_EOS)
pn_data_put_map(properties);
if (pn_data_type(properties) == PN_MAP) {
static pn_bytes_t _exclusive = { 9, "exclusive" };
pn_data_enter(properties);
pn_data_put_symbol(properties, _exclusive);
pn_data_put_bool(properties, true);
pn_data_exit(properties);
}
else
pn_logf("%s: unexpected TERMINUS PROPERTIES type [%d], MAP expected",
__func__, pn_data_type(properties));
pn_data_rewind(properties);
} else {
pn_terminus_set_address(pn_link_target(link), name);
pn_terminus_set_address(pn_link_source(link), name);
}
And after that I have seen that my temporary queues started to be created
in exclusive mode, at least qpid-stat shows this:
queuedur autoDel excl msg msgIn
msgOut bytes bytesIn bytesOut cons bind
==
21948F78- . . . -70568712B_receiver-27663.0 YY1 1
0 4354350 1 1
But after clien program that has created this queue termionated, the queue
regrettably
was not deleted, but "exclusive" property was dropped instead:
queuedur autoDel excl msg msgIn
msgOut bytes bytesIn bytesOut cons bind
==
21948F78- . . . -70568712B_receiver-27663.0 Y 0 1
1 0435 435 0 1
Setting lifetime-property to delete-on-close in same way also does not make
any difference.
Any hope to get temporary queue deleted without terminating server (responder)
process?
Best regards,
On 04.07.2018 11:26, Gordon Sim wrote:
> On 03/07/18 21:15, Michael Ivanov wrote:
>> Hallo,
>>
>> I'm trying to create a queue using qmf message to qpidd cpp (1.38) as follows
>> (pseudocode, actually pn_data_* functions are used to construct the message):
>>
>> subject='broker',
>> correlation_id='1',
>> reply_to='X',
>> properties={
>> 'method': 'request'
>> 'qmf.opcode': '_method_request',
>> 'x-amqp-0-10.app-id': 'qmf2',
>> },
>> content={
>> '_object_id': {
>> '_object_name': 'org.apache.qpid.broker:broker:amqp-broker'
>> },
>> '_method_name': 'create',
>> '_arguments': {
>> 'type': 'queue',
>> 'name': u'test',
>> 'strict': True,
>> 'properties': {
>> 'auto-delete': True,
>> }
>> }
>> }
>>
>> Queue is created and auto-delete property is set, but as I understand to make
>> auto deletion actually work 'exclusive' property has to be added. Is it
>> possible using this kind of message? I tried to add 'exclusive': True to
>> message but it did not work.
>
> Try setting :
>
> 'qpid.lifetime-policy': 'delete-on-close'
>
> in the properties?
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
--
\ / | |
(OvO) | Михаил Иванов|
(^^^) | Тел.:+7(911) 223-1300|
\^/ | E-mail: iv...@isle.spb.ru |
^ ^ | |
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
On 03/07/18 21:15, Michael Ivanov wrote:
Hallo,
I'm trying to create a queue using qmf message to qpidd cpp (1.38) as follows
(pseudocode, actually pn_data_* functions are used to construct the message):
subject='broker',
correlation_id='1',
reply_to='X',
properties={
'method': 'request'
'qmf.opcode': '_method_request',
'x-amqp-0-10.app-id': 'qmf2',
},
content={
'_object_id': {
'_object_name': 'org.apache.qpid.broker:broker:amqp-broker'
},
'_method_name': 'create',
'_arguments': {
'type': 'queue',
'name': u'test',
'strict': True,
'properties': {
'auto-delete': True,
}
}
}
Queue is created and auto-delete property is set, but as I understand to make
auto deletion actually work 'exclusive' property has to be added. Is it
possible using this kind of message? I tried to add 'exclusive': True to
message but it did not work.
Try setting :
'qpid.lifetime-policy': 'delete-on-close'
in the properties?
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
Hallo,
I'm trying to create a queue using qmf message to qpidd cpp (1.38) as follows
(pseudocode, actually pn_data_* functions are used to construct the message):
subject='broker',
correlation_id='1',
reply_to='X',
properties={
'method': 'request'
'qmf.opcode': '_method_request',
'x-amqp-0-10.app-id': 'qmf2',
},
content={
'_object_id': {
'_object_name': 'org.apache.qpid.broker:broker:amqp-broker'
},
'_method_name': 'create',
'_arguments': {
'type': 'queue',
'name': u'test',
'strict': True,
'properties': {
'auto-delete': True,
}
}
}
Queue is created and auto-delete property is set, but as I understand to make
auto deletion actually work 'exclusive' property has to be added. Is it
possible using this kind of message? I tried to add 'exclusive': True to
message but it did not work.
Best regards,
On 13.03.2018 13:41, Gordon Sim wrote:
> On 13/03/18 10:25, Michael Ivanov wrote:
>> Hallo,
>>
>> On 16.02.2018 21:37, Gordon Sim wrote:
>>
>>> If the queue is both auto-delete and exclusive, it is deleted when the link
>>> that caused it to be created is closed. If it is
>>> auto-delete only, it is deleted when it is 'no longer used', which is
>>> currently interpreted by qpidd to being when there are no
>>> links to it in either direction.In AMQP 1.0 this can also be controlled
>>> through the 'lifetime-policy' property in the
>>> dynamic-node-properties field.
>>>
>>> Unfortunately the messenger API gives you no control over the type of queue
>>> you request, nor (I think) does it allow you to
>>> close senders (which may in itself be a problem) or use an anonymous sender.
>>
>> Does proton have alternate interface which would allow to specify type of
>> temporary queue or lifetime policy?
>
> Yes, the recommended API for proton provides better controller over the
> various aspects of the protocol than the messenger API
> did: https://qpid.apache.org/proton/index.html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
--
\ / | |
(OvO) | Михаил Иванов|
(^^^) | Тел.:+7(911) 223-1300|
\^/ | E-mail: iv...@isle.spb.ru |
^ ^ | |
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
On 13/03/18 10:25, Michael Ivanov wrote: Hallo, On 16.02.2018 21:37, Gordon Sim wrote: If the queue is both auto-delete and exclusive, it is deleted when the link that caused it to be created is closed. If it is auto-delete only, it is deleted when it is 'no longer used', which is currently interpreted by qpidd to being when there are no links to it in either direction.In AMQP 1.0 this can also be controlled through the 'lifetime-policy' property in the dynamic-node-properties field. Unfortunately the messenger API gives you no control over the type of queue you request, nor (I think) does it allow you to close senders (which may in itself be a problem) or use an anonymous sender. Does proton have alternate interface which would allow to specify type of temporary queue or lifetime policy? Yes, the recommended API for proton provides better controller over the various aspects of the protocol than the messenger API did: https://qpid.apache.org/proton/index.html - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
Hallo, On 16.02.2018 21:37, Gordon Sim wrote: > If the queue is both auto-delete and exclusive, it is deleted when the link > that caused it to be created is closed. If it is > auto-delete only, it is deleted when it is 'no longer used', which is > currently interpreted by qpidd to being when there are no > links to it in either direction.In AMQP 1.0 this can also be controlled > through the 'lifetime-policy' property in the > dynamic-node-properties field. > > Unfortunately the messenger API gives you no control over the type of queue > you request, nor (I think) does it allow you to > close senders (which may in itself be a problem) or use an anonymous sender. Does proton have alternate interface which would allow to specify type of temporary queue or lifetime policy? Best regards, -- Michael Ivanov - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
On 15/02/18 20:40, Michael Ivanov wrote: Greetings! I have a problem with autodeleted queues. I have a service that reads amqp messages and sends replies using reply_to value. Client that sends the message generates temporary queue to use for replies using "/#" syntax. Client sends the message, waits for reply and terminates. But temporary queue, created by client, still remains in the broker until the other end (my service) is restarted. Here's how this queue is shown by qpid-stat after client has terminated: queue dur autoDel excl msg msgIn msgOut bytes bytesIn bytesOut cons bind === 821 857-349.0 Y 0 1 1 0257 257 01 Queue is marked as auto delete, number of connections is 0, but bind is 1. I guess this is the reason why it is not deleted. Is there a way to request messenger to unbind the queue as soon as the message is sent? I am using pn_messenger_* interface of proton library. I believe the problem is that there is still a sending link to the queue(s). (This doesn't show up in the qpid-stat output because it was defined for older iterations of AMQP where there was no such concept). If the queue is both auto-delete and exclusive, it is deleted when the link that caused it to be created is closed. If it is auto-delete only, it is deleted when it is 'no longer used', which is currently interpreted by qpidd to being when there are no links to it in either direction. In AMQP 1.0 this can also be controlled through the 'lifetime-policy' property in the dynamic-node-properties field. Unfortunately the messenger API gives you no control over the type of queue you request, nor (I think) does it allow you to close senders (which may in itself be a problem) or use an anonymous sender. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Temporary queues deletion
On 15/02/18 20:40, Michael Ivanov wrote: Greetings! I have a problem with autodeleted queues. I have a service that reads amqp messages and sends replies using reply_to value. Client that sends the message generates temporary queue to use for replies using "/#" syntax. Client sends the message, waits for reply and terminates. But temporary queue, created by client, still remains in the broker until the other end (my service) is restarted. Here's how this queue is shown by qpid-stat after client has terminated: queue dur autoDel excl msg msgIn msgOut bytes bytesIn bytesOut cons bind === 821 857-349.0 Y 0 1 1 0257 257 01 Queue is marked as auto delete, number of connections is 0, but bind is 1. I guess this is the reason why it is not deleted. Do you have a simple reproducer? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
