Re: Proposed Feature Removal from Dispatch Router

[Ken Giusti]

On Mon, Apr 16, 2018 at 1:06 PM, Alan Conway  wrote:
> On Mon, Apr 16, 2018 at 12:39 PM, Ken Giusti  wrote:
>
>> On Mon, Apr 16, 2018 at 12:11 PM, Alan Conway  wrote:
>> > On Mon, Apr 16, 2018 at 11:00 AM, Ken Giusti  wrote:
>> >
>> >> Yeah,  exactly.
>> >>
>> >> It's as if you applied a priority to each disposition in the following
>> >> order (highest first):
>> >> REJECTED
>> >> ACCEPTED
>> >> MODIFIED
>> >> RELEASED
>> >>
>> >> The router returns the highest priority disposition from all
>> >> consumer's returned dispositions.
>> >>
>> >>
>> > What if some consumer never returns a disposition?
>>
>> Right - or classic 'slow consumer'.   Without some sort of timeout
>> mechanism the transfer would stall indefinitely.
>> But doesn't the same apply for unicast?
>>
>> In the oslo.messaging driver, all message operations have a timeout
>> and TTL.  In that
>> case the sender would abort and drop the link.  Will any application
>> expect to wait forever?
>> Hold on - I meant to say "any well designed application"  ;)
>>
>>
>> > What if all consumers never return a disposition?
>>
>> Same deal.
>>
>> > What if there are no consumers?
>>
>> We have that now - credit is never granted and a sender can block
>> indefinitely.
>>
>
> What is the use case for this? If I cared about the disposition of a
> message for multiple receivers, I'd send it on multiple unicast addresses
> so I know what happened on each one. If I didn't care, I'd send multicast
> and pre-settled and genuinely not care.

You and I both.

But the original question I posited is

"what does a user/app expect if they send multicast unsettled?"

I'm trying to understand what people expect to happen when they do this.

Right now we provide two solutions, one to explicitly fail unless they
set the "no, I know what I'm doing" switch.
In that case, we settle it locally, which is a subtle fake that may
lead to unanticipated behaviors.

> Multicast is very useful when some
> unknown number of receivers, possibly zero, can subscribe based on interest
> - but the sender doesn't know or care how many there are. What's the use
> case where the sender must know that the message was received by somebody
> but doesn't care who or how many?

I can't see one honestly.   If no one else does then the correct
behavior should always
be REJECT, since anything else is undefined.



-- 
-K

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Proposed Feature Removal from Dispatch Router

[Alan Conway]

On Mon, Apr 16, 2018 at 1:50 PM, Gordon Sim  wrote:

> On 16/04/18 15:24, Ken Giusti wrote:
>
>> To reply to my own question:
>>
>> IMHO when sending an unsettled multicast I would expect
>> 1) that all present consumers will get a copy of the message and:
>> 2) that any potential consumers that are *not* present would not get a
>> copy of the message (right, that's a no-brainer, but hear me out).
>> 3) if any consumer signals a REJECT
>>
>> So I would like the router to:
>>
>> 1) send back a final disposition of REJECT if *any* client returned a
>> REJECT.
>> The spec is pretty clear that the message is considered invalid by the
>> recipient
>>   in this case.  That's a pretty big deal, since I assumed that the
>> message is
>> not invalid when it was sent.  This could possibly indicate a bug or a
>> state
>> mismatch between sender and receiver.  I would want to know about this.
>>
>
> What if there are 10 consumers, and only one of them rejects it? Clearly
> there is a problem, but is it the sender that is best able to react to
> that? Perhaps the consumer that rejected it is at fault since all the other
> consumers considered the message valid.
>
> What if two of the consumer reject for different reasons (i.e. with
> different errors)?
>
> While I agree that the rejection is important information, I'm not sure
> that propagating it to the sender is the necessarily the most useful way of
> signalling this. Maybe some eventing scheme would actually be useful,
> allowing the system to configure where to direct the information so it can
> be acted upon. Failing that better control over logging of this sort of
> thing.
>
>
>
This is why I feel like multicast-with-settlement is not really a useful
feature. If you need to know what happens at the receivers then you need to
address them individually - we don't need a complicated scheme for trying
to jam multiple dispositions into one - the user needs to get multiple
dispositions via simple unicast addresses. If you need decoupled messaging
with store-forward delivery guarantees, then you need a broker. To me, the
only time router multicast is useful is when you don't care about
dispositions, and you want to send pre-settled messages on a best-effort
basis to an unknown (possibly empty) set of receivers.

Broker-J BDB JE High Availability Time Sync issue

[Bryan Dixon]

We are using Broker-J 7.0.2 and just ran into a Berkeley HA Time Sync issue
that I'm wondering if anyone else has run into.  The stackTrace is at the
end of this post.   We are running on Windows Server 2012 R2 6.3 amd64 and
our JDK is Oracle Corporation 1.8.0_162-b12.  We have 3 servers as part of
our HA setup.

This error occurred in our production environment which has been live for
just a couple of weeks.  We never ran into this in our Test or Dev
environments that have been running for a few months.   When one of our
admins checked the clock times of all 3 servers they were completely in
sync.  Another admin stated that the server clock times are synced with NTP. 
Unfortunately our log files rolled off and I don't know exactly when this
error first occurred because the older log file are gone.

2018-04-16 04:10:57,039 ERROR [Group-Change-Learner:prodbroker:prodbroker2]
(o.a.q.s.u.ServerScopedRuntimeException) - Exception on master check
com.sleepycat.je.EnvironmentFailureException: (JE 7.4.5) Environment must be
closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment
invalid because of previous exception: (JE 7.4.5)
prodbroker2(2):D:\qpidwork\prodbroker2\config Clock delta: 8859 ms. between
Feeder: prodbroker1 and this Replica exceeds max permissible delta: 2000 ms.
HANDSHAKE_ERROR: Error during the handshake between two nodes. Some validity
or compatibility check failed, preventing further communication between the
nodes. Environment is invalid and must be closed. Originally thrown by HA
thread: UNKNOWN prodbroker2(2) Originally thrown by HA thread: UNKNOWN
prodbroker2(2)
at
com.sleepycat.je.EnvironmentFailureException.wrapSelf(EnvironmentFailureException.java:228)
at
com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1766)
at
com.sleepycat.je.dbi.EnvironmentImpl.checkOpen(EnvironmentImpl.java:1775)
at com.sleepycat.je.Environment.checkOpen(Environment.java:2473)
at com.sleepycat.je.DbInternal.checkOpen(DbInternal.java:105)
at
com.sleepycat.je.rep.ReplicatedEnvironment.checkOpen(ReplicatedEnvironment.java:1052)
at
com.sleepycat.je.rep.ReplicatedEnvironment.getState(ReplicatedEnvironment.java:764)
at
org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade$RemoteNodeStateLearner.executeDatabasePingerOnNodeChangesIfMaster(ReplicatedEnvironmentFacade.java:2276)
at
org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade$RemoteNodeStateLearner.call(ReplicatedEnvironmentFacade.java:2042)
at
org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade$RemoteNodeStateLearner.call(ReplicatedEnvironmentFacade.java:2012)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid
because of previous exception: (JE 7.4.5)
prodbroker2(2):D:\qpidwork\prodbroker2\config Clock delta: 8859 ms. between
Feeder: prodbroker1 and this Replica exceeds max permissible delta: 2000 ms.
HANDSHAKE_ERROR: Error during the handshake between two nodes. Some validity
or compatibility check failed, preventing further communication between the
nodes. Environment is invalid and must be closed. Originally thrown by HA
thread: UNKNOWN prodbroker2(2) Originally thrown by HA thread: UNKNOWN
prodbroker2(2)
at
com.sleepycat.je.rep.stream.ReplicaFeederHandshake.checkClockSkew(ReplicaFeederHandshake.java:424)
at
com.sleepycat.je.rep.stream.ReplicaFeederHandshake.execute(ReplicaFeederHandshake.java:261)
at 
com.sleepycat.je.rep.impl.node.Replica.initReplicaLoop(Replica.java:691)
at
com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:474)
at 
com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:409)
at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:1873)



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Proposed Feature Removal from Dispatch Router

[Gordon Sim]

On 16/04/18 15:24, Ken Giusti wrote:

To reply to my own question:

IMHO when sending an unsettled multicast I would expect
1) that all present consumers will get a copy of the message and:
2) that any potential consumers that are *not* present would not get a
copy of the message (right, that's a no-brainer, but hear me out).
3) if any consumer signals a REJECT

So I would like the router to:

1) send back a final disposition of REJECT if *any* client returned a REJECT.
The spec is pretty clear that the message is considered invalid by the recipient
  in this case.  That's a pretty big deal, since I assumed that the message is
not invalid when it was sent.  This could possibly indicate a bug or a state
mismatch between sender and receiver.  I would want to know about this.


What if there are 10 consumers, and only one of them rejects it? Clearly 
there is a problem, but is it the sender that is best able to react to 
that? Perhaps the consumer that rejected it is at fault since all the 
other consumers considered the message valid.


What if two of the consumer reject for different reasons (i.e. with 
different errors)?


While I agree that the rejection is important information, I'm not sure 
that propagating it to the sender is the necessarily the most useful way 
of signalling this. Maybe some eventing scheme would actually be useful, 
allowing the system to configure where to direct the information so it 
can be acted upon. Failing that better control over logging of this sort 
of thing.


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Proposed Feature Removal from Dispatch Router

[Gordon Sim]

On 16/04/18 17:39, Ken Giusti wrote:

On Mon, Apr 16, 2018 at 12:11 PM, Alan Conway  wrote:

On Mon, Apr 16, 2018 at 11:00 AM, Ken Giusti  wrote:


Yeah,  exactly.

It's as if you applied a priority to each disposition in the following
order (highest first):
REJECTED
ACCEPTED
MODIFIED
RELEASED

The router returns the highest priority disposition from all
consumer's returned dispositions.



What if some consumer never returns a disposition?


Right - or classic 'slow consumer'.   Without some sort of timeout
mechanism the transfer would stall indefinitely.
But doesn't the same apply for unicast?


With multicast, all the messages go to all receivers, so a slow consumer 
will stall *all* transfers which eventually will prevent other consumers 
getting any more messages.


One use of multicast is for a pub-sub style communication pattern where 
you want the publishers and subscribers to be decoupled. Allowing one 
subscriber to bring message flow to a halt for publishers and all other 
subscribers might not be desirable in those cases.



In the oslo.messaging driver, all message operations have a timeout
and TTL.  In that
case the sender would abort and drop the link.  Will any application
expect to wait forever?
Hold on - I meant to say "any well designed application"  ;)



What if all consumers never return a disposition?


Same deal.


What if there are no consumers?


We have that now - credit is never granted and a sender can block indefinitely.


That is how anycast works now (if you already have credit then the 
messages are released). I don't think there is any credit propagation 
for multicast at present is there?


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Proposed Feature Removal from Dispatch Router

[Alan Conway]

On Mon, Apr 16, 2018 at 12:39 PM, Ken Giusti  wrote:

> On Mon, Apr 16, 2018 at 12:11 PM, Alan Conway  wrote:
> > On Mon, Apr 16, 2018 at 11:00 AM, Ken Giusti  wrote:
> >
> >> Yeah,  exactly.
> >>
> >> It's as if you applied a priority to each disposition in the following
> >> order (highest first):
> >> REJECTED
> >> ACCEPTED
> >> MODIFIED
> >> RELEASED
> >>
> >> The router returns the highest priority disposition from all
> >> consumer's returned dispositions.
> >>
> >>
> > What if some consumer never returns a disposition?
>
> Right - or classic 'slow consumer'.   Without some sort of timeout
> mechanism the transfer would stall indefinitely.
> But doesn't the same apply for unicast?
>
> In the oslo.messaging driver, all message operations have a timeout
> and TTL.  In that
> case the sender would abort and drop the link.  Will any application
> expect to wait forever?
> Hold on - I meant to say "any well designed application"  ;)
>
>
> > What if all consumers never return a disposition?
>
> Same deal.
>
> > What if there are no consumers?
>
> We have that now - credit is never granted and a sender can block
> indefinitely.
>

What is the use case for this? If I cared about the disposition of a
message for multiple receivers, I'd send it on multiple unicast addresses
so I know what happened on each one. If I didn't care, I'd send multicast
and pre-settled and genuinely not care. Multicast is very useful when some
unknown number of receivers, possibly zero, can subscribe based on interest
- but the sender doesn't know or care how many there are. What's the use
case where the sender must know that the message was received by somebody
but doesn't care who or how many?

Re: Proposed Feature Removal from Dispatch Router

[Ken Giusti]

On Mon, Apr 16, 2018 at 12:11 PM, Alan Conway  wrote:
> On Mon, Apr 16, 2018 at 11:00 AM, Ken Giusti  wrote:
>
>> Yeah,  exactly.
>>
>> It's as if you applied a priority to each disposition in the following
>> order (highest first):
>> REJECTED
>> ACCEPTED
>> MODIFIED
>> RELEASED
>>
>> The router returns the highest priority disposition from all
>> consumer's returned dispositions.
>>
>>
> What if some consumer never returns a disposition?

Right - or classic 'slow consumer'.   Without some sort of timeout
mechanism the transfer would stall indefinitely.
But doesn't the same apply for unicast?

In the oslo.messaging driver, all message operations have a timeout
and TTL.  In that
case the sender would abort and drop the link.  Will any application
expect to wait forever?
Hold on - I meant to say "any well designed application"  ;)


> What if all consumers never return a disposition?

Same deal.

> What if there are no consumers?

We have that now - credit is never granted and a sender can block indefinitely.


>
>
>
>>
>>
>> -K
>>
>>
>> On Mon, Apr 16, 2018 at 10:51 AM, Michael Goulish 
>> wrote:
>> > You mean your rules to be applied exclusively, and in that order, right?
>> > i.e.
>> >
>> > if ( anybody rejected )
>> > {
>> >   disposition = rejected
>> > }
>> > *else*
>> > if ( anybody accepted )
>> > {
>> >   disposition = accepted
>> > }
>> >
>> >
>> >
>> >
>> > On Mon, Apr 16, 2018 at 10:24 AM, Ken Giusti  wrote:
>> >
>> >> To reply to my own question:
>> >>
>> >> IMHO when sending an unsettled multicast I would expect
>> >> 1) that all present consumers will get a copy of the message and:
>> >> 2) that any potential consumers that are *not* present would not get a
>> >> copy of the message (right, that's a no-brainer, but hear me out).
>> >> 3) if any consumer signals a REJECT
>> >>
>> >> So I would like the router to:
>> >>
>> >> 1) send back a final disposition of REJECT if *any* client returned a
>> >> REJECT.
>> >> The spec is pretty clear that the message is considered invalid by the
>> >> recipient
>> >>  in this case.  That's a pretty big deal, since I assumed that the
>> message
>> >> is
>> >> not invalid when it was sent.  This could possibly indicate a bug or a
>> >> state
>> >> mismatch between sender and receiver.  I would want to know about this.
>> >>
>> >> 2) send back a final disposition of ACCEPTED if at least one client
>> >> ACCEPTED.  Ignore MODIFIED and RELEASED in this case, since
>> >> 2a) RELEASED indicates we can resend safely, which we cannot
>> >> (someone ACCEPTED)
>> >> 2b) MODIFIED is in doubt, also cannot resend safely and I feel can
>> >> be considered as an equivalent case as #2 above
>> >>
>> >> 3) Otherwise for a mix of MODIFIED and RELEASED return MODIFIED as we
>> >> cannot re-send the same message.
>> >> 4) else all RELEASED, so return RELEASED.
>> >>
>> >>
>> >>
>> >> Again, this is MHO and I only present it as a strawman for
>> >> consideration and discussion.  I'm not convinced holding state in the
>> >> router while it waits
>> >> for all consumers to reply is practical (or desired in the slow consumer
>> >> case).
>> >>
>> >>
>> >> -K
>> >>
>> >> On Mon, Apr 16, 2018 at 9:49 AM, Ken Giusti  wrote:
>> >> > We really should try to do something smarter in the case of unsettled
>> >> > multicast rather than either of the current approaches.
>> >> >
>> >> > What does an application/dev expect when it sends any message
>> >> > unsettled?  It expects to block until eventually it gets some
>> >> > indication of whether or not the message was delivered as intended.
>> >> > In the case of single consumer the expectation is obvious and well
>> >> > handled by the router.
>> >> >
>> >> > But in the case of multicast it is a different story: here we have the
>> >> > possibility that the message may be both consumed by one recipient and
>> >> > rejected by another.  So the question is: from the POV of the dev/app,
>> >> > what is the "obvious" default action the router should perform in that
>> >> > case?
>> >> >
>> >> > -K
>> >> >
>> >> >
>> >> > On Fri, Apr 13, 2018 at 3:38 PM, Chuck Rolke 
>> wrote:
>> >> >> I would prefer to keep the feature enforced as it is now. I was one
>> who
>> >> >> was surprised to have a sender whose message is settled by the router
>> >> >> only to find out that it was not delivered anywhere.
>> >> >>
>> >> >> The document https://qpid.apache.org/releases/qpid-dispatch-1.0.0/
>> >> book/book.html#routing-patterns
>> >> >> needs to have a clearer explanation of the lossy nature of multicast
>> >> >> distribution.
>> >> >>
>> >> >> - Original Message -
>> >> >>> From: "Ted Ross" 
>> >> >>> To: users@qpid.apache.org
>> >> >>> Sent: Thursday, April 12, 2018 6:26:34 PM
>> >> >>> Subject: Re: Proposed Feature Removal from Dispatch Router
>> >> >>>
>> >> >>> For the record, here is the Jira for 

Re: Proposed Feature Removal from Dispatch Router

[Alan Conway]

On Mon, Apr 16, 2018 at 11:00 AM, Ken Giusti  wrote:

> Yeah,  exactly.
>
> It's as if you applied a priority to each disposition in the following
> order (highest first):
> REJECTED
> ACCEPTED
> MODIFIED
> RELEASED
>
> The router returns the highest priority disposition from all
> consumer's returned dispositions.
>
>
What if some consumer never returns a disposition?
What if all consumers never return a disposition?
What if there are no consumers?



>
>
> -K
>
>
> On Mon, Apr 16, 2018 at 10:51 AM, Michael Goulish 
> wrote:
> > You mean your rules to be applied exclusively, and in that order, right?
> > i.e.
> >
> > if ( anybody rejected )
> > {
> >   disposition = rejected
> > }
> > *else*
> > if ( anybody accepted )
> > {
> >   disposition = accepted
> > }
> >
> >
> >
> >
> > On Mon, Apr 16, 2018 at 10:24 AM, Ken Giusti  wrote:
> >
> >> To reply to my own question:
> >>
> >> IMHO when sending an unsettled multicast I would expect
> >> 1) that all present consumers will get a copy of the message and:
> >> 2) that any potential consumers that are *not* present would not get a
> >> copy of the message (right, that's a no-brainer, but hear me out).
> >> 3) if any consumer signals a REJECT
> >>
> >> So I would like the router to:
> >>
> >> 1) send back a final disposition of REJECT if *any* client returned a
> >> REJECT.
> >> The spec is pretty clear that the message is considered invalid by the
> >> recipient
> >>  in this case.  That's a pretty big deal, since I assumed that the
> message
> >> is
> >> not invalid when it was sent.  This could possibly indicate a bug or a
> >> state
> >> mismatch between sender and receiver.  I would want to know about this.
> >>
> >> 2) send back a final disposition of ACCEPTED if at least one client
> >> ACCEPTED.  Ignore MODIFIED and RELEASED in this case, since
> >> 2a) RELEASED indicates we can resend safely, which we cannot
> >> (someone ACCEPTED)
> >> 2b) MODIFIED is in doubt, also cannot resend safely and I feel can
> >> be considered as an equivalent case as #2 above
> >>
> >> 3) Otherwise for a mix of MODIFIED and RELEASED return MODIFIED as we
> >> cannot re-send the same message.
> >> 4) else all RELEASED, so return RELEASED.
> >>
> >>
> >>
> >> Again, this is MHO and I only present it as a strawman for
> >> consideration and discussion.  I'm not convinced holding state in the
> >> router while it waits
> >> for all consumers to reply is practical (or desired in the slow consumer
> >> case).
> >>
> >>
> >> -K
> >>
> >> On Mon, Apr 16, 2018 at 9:49 AM, Ken Giusti  wrote:
> >> > We really should try to do something smarter in the case of unsettled
> >> > multicast rather than either of the current approaches.
> >> >
> >> > What does an application/dev expect when it sends any message
> >> > unsettled?  It expects to block until eventually it gets some
> >> > indication of whether or not the message was delivered as intended.
> >> > In the case of single consumer the expectation is obvious and well
> >> > handled by the router.
> >> >
> >> > But in the case of multicast it is a different story: here we have the
> >> > possibility that the message may be both consumed by one recipient and
> >> > rejected by another.  So the question is: from the POV of the dev/app,
> >> > what is the "obvious" default action the router should perform in that
> >> > case?
> >> >
> >> > -K
> >> >
> >> >
> >> > On Fri, Apr 13, 2018 at 3:38 PM, Chuck Rolke 
> wrote:
> >> >> I would prefer to keep the feature enforced as it is now. I was one
> who
> >> >> was surprised to have a sender whose message is settled by the router
> >> >> only to find out that it was not delivered anywhere.
> >> >>
> >> >> The document https://qpid.apache.org/releases/qpid-dispatch-1.0.0/
> >> book/book.html#routing-patterns
> >> >> needs to have a clearer explanation of the lossy nature of multicast
> >> >> distribution.
> >> >>
> >> >> - Original Message -
> >> >>> From: "Ted Ross" 
> >> >>> To: users@qpid.apache.org
> >> >>> Sent: Thursday, April 12, 2018 6:26:34 PM
> >> >>> Subject: Re: Proposed Feature Removal from Dispatch Router
> >> >>>
> >> >>> For the record, here is the Jira for the feature in question:
> >> >>>
> >> >>> https://issues.apache.org/jira/browse/DISPATCH-744
> >> >>>
> >> >>> On Thu, Apr 12, 2018 at 6:20 PM, Ted Ross  wrote:
> >> >>> > We added a feature back in 1.0.0 to reject unsettled deliveries to
> >> >>> > multicast addresses by default.  This can be disabled through
> >> >>> > configuration but is on by default.
> >> >>> >
> >> >>> > The rationale was that the router would accept and settle
> unsettled
> >> >>> > multicasts even though it might not have delivered the messages to
> >> any
> >> >>> > consumer.  The rejection with error code was intended to inform
> users
> >> >>> > that they should pre-settle deliveries to 

Re: Proposed Feature Removal from Dispatch Router

[Ken Giusti]

Yeah,  exactly.

It's as if you applied a priority to each disposition in the following
order (highest first):
REJECTED
ACCEPTED
MODIFIED
RELEASED

The router returns the highest priority disposition from all
consumer's returned dispositions.



-K


On Mon, Apr 16, 2018 at 10:51 AM, Michael Goulish  wrote:
> You mean your rules to be applied exclusively, and in that order, right?
> i.e.
>
> if ( anybody rejected )
> {
>   disposition = rejected
> }
> *else*
> if ( anybody accepted )
> {
>   disposition = accepted
> }
>
>
>
>
> On Mon, Apr 16, 2018 at 10:24 AM, Ken Giusti  wrote:
>
>> To reply to my own question:
>>
>> IMHO when sending an unsettled multicast I would expect
>> 1) that all present consumers will get a copy of the message and:
>> 2) that any potential consumers that are *not* present would not get a
>> copy of the message (right, that's a no-brainer, but hear me out).
>> 3) if any consumer signals a REJECT
>>
>> So I would like the router to:
>>
>> 1) send back a final disposition of REJECT if *any* client returned a
>> REJECT.
>> The spec is pretty clear that the message is considered invalid by the
>> recipient
>>  in this case.  That's a pretty big deal, since I assumed that the message
>> is
>> not invalid when it was sent.  This could possibly indicate a bug or a
>> state
>> mismatch between sender and receiver.  I would want to know about this.
>>
>> 2) send back a final disposition of ACCEPTED if at least one client
>> ACCEPTED.  Ignore MODIFIED and RELEASED in this case, since
>> 2a) RELEASED indicates we can resend safely, which we cannot
>> (someone ACCEPTED)
>> 2b) MODIFIED is in doubt, also cannot resend safely and I feel can
>> be considered as an equivalent case as #2 above
>>
>> 3) Otherwise for a mix of MODIFIED and RELEASED return MODIFIED as we
>> cannot re-send the same message.
>> 4) else all RELEASED, so return RELEASED.
>>
>>
>>
>> Again, this is MHO and I only present it as a strawman for
>> consideration and discussion.  I'm not convinced holding state in the
>> router while it waits
>> for all consumers to reply is practical (or desired in the slow consumer
>> case).
>>
>>
>> -K
>>
>> On Mon, Apr 16, 2018 at 9:49 AM, Ken Giusti  wrote:
>> > We really should try to do something smarter in the case of unsettled
>> > multicast rather than either of the current approaches.
>> >
>> > What does an application/dev expect when it sends any message
>> > unsettled?  It expects to block until eventually it gets some
>> > indication of whether or not the message was delivered as intended.
>> > In the case of single consumer the expectation is obvious and well
>> > handled by the router.
>> >
>> > But in the case of multicast it is a different story: here we have the
>> > possibility that the message may be both consumed by one recipient and
>> > rejected by another.  So the question is: from the POV of the dev/app,
>> > what is the "obvious" default action the router should perform in that
>> > case?
>> >
>> > -K
>> >
>> >
>> > On Fri, Apr 13, 2018 at 3:38 PM, Chuck Rolke  wrote:
>> >> I would prefer to keep the feature enforced as it is now. I was one who
>> >> was surprised to have a sender whose message is settled by the router
>> >> only to find out that it was not delivered anywhere.
>> >>
>> >> The document https://qpid.apache.org/releases/qpid-dispatch-1.0.0/
>> book/book.html#routing-patterns
>> >> needs to have a clearer explanation of the lossy nature of multicast
>> >> distribution.
>> >>
>> >> - Original Message -
>> >>> From: "Ted Ross" 
>> >>> To: users@qpid.apache.org
>> >>> Sent: Thursday, April 12, 2018 6:26:34 PM
>> >>> Subject: Re: Proposed Feature Removal from Dispatch Router
>> >>>
>> >>> For the record, here is the Jira for the feature in question:
>> >>>
>> >>> https://issues.apache.org/jira/browse/DISPATCH-744
>> >>>
>> >>> On Thu, Apr 12, 2018 at 6:20 PM, Ted Ross  wrote:
>> >>> > We added a feature back in 1.0.0 to reject unsettled deliveries to
>> >>> > multicast addresses by default.  This can be disabled through
>> >>> > configuration but is on by default.
>> >>> >
>> >>> > The rationale was that the router would accept and settle unsettled
>> >>> > multicasts even though it might not have delivered the messages to
>> any
>> >>> > consumer.  The rejection with error code was intended to inform users
>> >>> > that they should pre-settle deliveries to multicast addresses in
>> >>> > keeping with the best-effort nature of multicast routing.
>> >>> >
>> >>> > In practice, this is more of an annoyance because none of the example
>> >>> > clients (and apparently the users' clients) actually do anything with
>> >>> > the error code in the rejected delivery.  The router appears to
>> >>> > silently drop such messages for no good reason and good will is
>> wasted
>> >>> > in chasing down the issue to "oh, you should turn off 

Re: Qpid Dispatch authenticate through ldap, is this possible

[mlange]

Ganesh Murthy wrote
> On Mon, Apr 16, 2018 at 10:08 AM, mlange 

> mlange@

>  wrote:
> 
>>
>> > That looks a bit as if artemis is trying to authenticate the connection
>> > via a client certificate. From the config snippet you supplied it
>> > doesn't look like it is using TLS, let alone supplying a client cert.
>> > Are you able to get a protocol trace for the interaction between the
>> > router and the broker? (A simple way to do this would be to start a
>> > router with that connector in with env var PN_TRACE_FRM=1 and capture
>> > the output)
>>
>> It shouldn't do that, trying to authenticate via client certificate
>> (well,
>> not yet at least)
>> With the same config, but then connecting directly to the broker (a
>> javax.jms.Connection(String user, String password); with the same
>> credentials) allows me to connect just fine.
>>
>> The trace gives quite some output; I think the relevant parts are these:
>> [0x7f595400bdb0]:  -> SASL
>> [0x7f595400bdb0]:  <- SASL
>> [0x7f595400bdb0]:0 <- @sasl-mechanisms(64)
>> [sasl-server-mechanisms=@PN_SYMBOL[:PLAIN, :ANONYMOUS]]
>> [0x7f595400bdb0]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS,
>> initial-response=b"

> anonymous@.host

> "]
>> [0x7f595400bdb0]:0 <- @sasl-outcome(68) [code=0]
>>
>> Here it seems as if qpid chooses to use ANONYMOUS to connect with the
>> broker
>> (which will not work, the broker is configured to require authentication)
>> whereas the broker seems to offer PLAIN as well.
>>
>> a bit later I see the connection:
>> [0x7f5954027d60]:4 <- @begin(17) [next-outgoing-id=0,
>> incoming-window=2147483647, outgoing-window=2147483647]
>> [0x7f5954027d60]:4 <- @attach(18)
>> [name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c7
>> 7f:1:1:1:testqueues.testqueue",
>> handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0,
>> source=@source(40) [address="ID:8b0bc583-315f-4f5
>> 4-8f17-ecc40379c77f:1:1:1",
>> durable=0, timeout=0, dynamic=false,
>> outcomes=@PN_SYMBOL[:"amqp:accepted:list", :"amqp:rejected:list",
>> :"amqp:released:list", :"amqp:modified:list"]], target=@target(41)
>> [address="testqueues.testqueue", durable=0, timeout=0, dynamic=false,
>> capabilities=@PN_SYMBOL[:queue]], initial-delivery-count=0,
>> max-message-size=0]
>> [0x7f5954027d60]:4 -> @begin(17) [remote-channel=4, next-outgoing-id=0,
>> incoming-window=2147483647, outgoing-window=2147483647]
>> [0x7f595400bdb0]:0 -> @begin(17) [next-outgoing-id=0,
>> incoming-window=2147483647, outgoing-window=2147483647]
>> [0x7f595400bdb0]:0 -> @attach(18)
>> [name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c7
>> 7f:1:1:1:testqueues.testqueue",
>> handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0,
>> source=@source(40) [address="ID:8b0bc583-315f-4f5
>> 4-8f17-ecc40379c77f:1:1:1",
>> durable=0, timeout=0, dynamic=false,
>> outcomes=@PN_SYMBOL[:"amqp:accepted:list", :"amqp:rejected:list",
>> :"amqp:released:list", :"amqp:modified:list"]], target=@target(41)
>> [address="testqueues.testqueue", durable=0, timeout=0, dynamic=false,
>> capabilities=@PN_SYMBOL[:queue]], initial-delivery-count=0,
>> max-message-size=0]
>> [0x7f595400bdb0]:0 <- @close(24) [error=@error(29)
>> [condition=:"amqp:internal-error", description="Unrecoverable error:
>> AMQ119031: Unable to validate user from /192.168.0.1:52202. Username:
>> null;
>> SSL certificate subject DN: unavailable"]]
>> [0x7f595400bdb0]:  <- EOS
>> [0x7f595400bdb0]:0 -> @close(24) []
>> [0x7f595400bdb0]:  -> EOS
>> [0x7f5954027d60]:4 -> @attach(18)
>> [name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c7
>> 7f:1:1:1:testqueues.testqueue",
>> handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0,
>> source=@source(40) [durable=0, timeout=0, dynamic=false],
>> target=@target(41)
>> [durable=0, timeout=0, dynamic=false], initial-delivery-count=0,
>> max-message-size=0]
>> [0x7f5954027d60]:4 -> @detach(22) [handle=0, closed=false,
>> error=@error(29)
>> [condition=:"qd:routed-link-lost", description="Connectivity to the peer
>> container was lost"]]
>> [0x7f5954027d60]:4 <- @detach(22) [handle=0, closed=true]
>>
>> Username is null, as well as client-certificates not provided (which is
>> logical, since there are none yet);
>>
>> When I add saslMechanisms: PLAIN to the connection{} I see a new error in
>> the SERVER log module (server.log):
>>  proton:io:sasl_error SASL(-4): no mechanism available: No worthy mechs
>> found (Authentication failed [mech=none])
>>
> Is it possible that you don't have the relevant cyrus-sasl-plain libraries
> installed? Does the tests/system_tests_sasl_plain.py pass for you? If you
> look at that test, you will notice that one router is trying to connect to
> another router using PLAIN mech.
> 
>>
>> which is weird, as it seems that PLAIN is offered by the broker. (or I am
>> interpreting things completely wrong)
>>
>>
>>
>> --
>> Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936
>> .html
>>
>> 

Re: QDR: transacted connections with multiple brokers

[Gordon Sim]

On 16/04/18 14:24, Alan Conway wrote:

On Mon, Apr 16, 2018 at 9:10 AM, mlange  wrote:

Agree with that one; a proper distributed TX would indeed be the best
solution; Couldn't find any issue in the Jira, are there plans to build
this? I think it would be a great addition.



No immediate plans that I'm aware of. It has always been a potential
feature but never at the top of the priority pile. By all means raise it,
so we can track interest and maybe encourage someone to contribute some
code.


Just for clarity, I don't see this (primarily) as a feature for the router.

A distributed transaction co-ordinator requires highly available state. 
It would therefore in my view be an additional component in the 
architecture (ideally based on some existing transaction coordinator).


Depending on the architecture of the solution, it might be that the 
router network does not need to do anything other than correctly 
propagate the transaction context along with transfers and dispositions 
(which it should already do) and perhaps route links for the 
co-ordinator (which again it can already do). If the co-ordinator was 
supposed to communicate with the resources through the router network, 
that might require some additional tweaks to the current router 
behaviour/feature set (at present I think it would have to use virtual 
hosts for each of the resources, with the link route for the controller 
defined appropriately).


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Qpid Dispatch authenticate through ldap, is this possible

[Ganesh Murthy]

On Mon, Apr 16, 2018 at 10:08 AM, mlange  wrote:

>
> > That looks a bit as if artemis is trying to authenticate the connection
> > via a client certificate. From the config snippet you supplied it
> > doesn't look like it is using TLS, let alone supplying a client cert.
> > Are you able to get a protocol trace for the interaction between the
> > router and the broker? (A simple way to do this would be to start a
> > router with that connector in with env var PN_TRACE_FRM=1 and capture
> > the output)
>
> It shouldn't do that, trying to authenticate via client certificate (well,
> not yet at least)
> With the same config, but then connecting directly to the broker (a
> javax.jms.Connection(String user, String password); with the same
> credentials) allows me to connect just fine.
>
> The trace gives quite some output; I think the relevant parts are these:
> [0x7f595400bdb0]:  -> SASL
> [0x7f595400bdb0]:  <- SASL
> [0x7f595400bdb0]:0 <- @sasl-mechanisms(64)
> [sasl-server-mechanisms=@PN_SYMBOL[:PLAIN, :ANONYMOUS]]
> [0x7f595400bdb0]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS,
> initial-response=b"anonym...@masterbroker.host.name"]
> [0x7f595400bdb0]:0 <- @sasl-outcome(68) [code=0]
>
> Here it seems as if qpid chooses to use ANONYMOUS to connect with the
> broker
> (which will not work, the broker is configured to require authentication)
> whereas the broker seems to offer PLAIN as well.
>
> a bit later I see the connection:
> [0x7f5954027d60]:4 <- @begin(17) [next-outgoing-id=0,
> incoming-window=2147483647, outgoing-window=2147483647]
> [0x7f5954027d60]:4 <- @attach(18)
> [name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c7
> 7f:1:1:1:testqueues.testqueue",
> handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0,
> source=@source(40) [address="ID:8b0bc583-315f-4f5
> 4-8f17-ecc40379c77f:1:1:1",
> durable=0, timeout=0, dynamic=false,
> outcomes=@PN_SYMBOL[:"amqp:accepted:list", :"amqp:rejected:list",
> :"amqp:released:list", :"amqp:modified:list"]], target=@target(41)
> [address="testqueues.testqueue", durable=0, timeout=0, dynamic=false,
> capabilities=@PN_SYMBOL[:queue]], initial-delivery-count=0,
> max-message-size=0]
> [0x7f5954027d60]:4 -> @begin(17) [remote-channel=4, next-outgoing-id=0,
> incoming-window=2147483647, outgoing-window=2147483647]
> [0x7f595400bdb0]:0 -> @begin(17) [next-outgoing-id=0,
> incoming-window=2147483647, outgoing-window=2147483647]
> [0x7f595400bdb0]:0 -> @attach(18)
> [name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c7
> 7f:1:1:1:testqueues.testqueue",
> handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0,
> source=@source(40) [address="ID:8b0bc583-315f-4f5
> 4-8f17-ecc40379c77f:1:1:1",
> durable=0, timeout=0, dynamic=false,
> outcomes=@PN_SYMBOL[:"amqp:accepted:list", :"amqp:rejected:list",
> :"amqp:released:list", :"amqp:modified:list"]], target=@target(41)
> [address="testqueues.testqueue", durable=0, timeout=0, dynamic=false,
> capabilities=@PN_SYMBOL[:queue]], initial-delivery-count=0,
> max-message-size=0]
> [0x7f595400bdb0]:0 <- @close(24) [error=@error(29)
> [condition=:"amqp:internal-error", description="Unrecoverable error:
> AMQ119031: Unable to validate user from /192.168.0.1:52202. Username:
> null;
> SSL certificate subject DN: unavailable"]]
> [0x7f595400bdb0]:  <- EOS
> [0x7f595400bdb0]:0 -> @close(24) []
> [0x7f595400bdb0]:  -> EOS
> [0x7f5954027d60]:4 -> @attach(18)
> [name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c7
> 7f:1:1:1:testqueues.testqueue",
> handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0,
> source=@source(40) [durable=0, timeout=0, dynamic=false],
> target=@target(41)
> [durable=0, timeout=0, dynamic=false], initial-delivery-count=0,
> max-message-size=0]
> [0x7f5954027d60]:4 -> @detach(22) [handle=0, closed=false, error=@error(29)
> [condition=:"qd:routed-link-lost", description="Connectivity to the peer
> container was lost"]]
> [0x7f5954027d60]:4 <- @detach(22) [handle=0, closed=true]
>
> Username is null, as well as client-certificates not provided (which is
> logical, since there are none yet);
>
> When I add saslMechanisms: PLAIN to the connection{} I see a new error in
> the SERVER log module (server.log):
>  proton:io:sasl_error SASL(-4): no mechanism available: No worthy mechs
> found (Authentication failed [mech=none])
>
Is it possible that you don't have the relevant cyrus-sasl-plain libraries
installed? Does the tests/system_tests_sasl_plain.py pass for you? If you
look at that test, you will notice that one router is trying to connect to
another router using PLAIN mech.

>
> which is weird, as it seems that PLAIN is offered by the broker. (or I am
> interpreting things completely wrong)
>
>
>
> --
> Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936
> .html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org

Re: Proposed Feature Removal from Dispatch Router

[Ken Giusti]

To reply to my own question:

IMHO when sending an unsettled multicast I would expect
1) that all present consumers will get a copy of the message and:
2) that any potential consumers that are *not* present would not get a
copy of the message (right, that's a no-brainer, but hear me out).
3) if any consumer signals a REJECT

So I would like the router to:

1) send back a final disposition of REJECT if *any* client returned a REJECT.
The spec is pretty clear that the message is considered invalid by the recipient
 in this case.  That's a pretty big deal, since I assumed that the message is
not invalid when it was sent.  This could possibly indicate a bug or a state
mismatch between sender and receiver.  I would want to know about this.

2) send back a final disposition of ACCEPTED if at least one client
ACCEPTED.  Ignore MODIFIED and RELEASED in this case, since
2a) RELEASED indicates we can resend safely, which we cannot
(someone ACCEPTED)
2b) MODIFIED is in doubt, also cannot resend safely and I feel can
be considered as an equivalent case as #2 above

3) Otherwise for a mix of MODIFIED and RELEASED return MODIFIED as we
cannot re-send the same message.
4) else all RELEASED, so return RELEASED.



Again, this is MHO and I only present it as a strawman for
consideration and discussion.  I'm not convinced holding state in the
router while it waits
for all consumers to reply is practical (or desired in the slow consumer case).


-K

On Mon, Apr 16, 2018 at 9:49 AM, Ken Giusti  wrote:
> We really should try to do something smarter in the case of unsettled
> multicast rather than either of the current approaches.
>
> What does an application/dev expect when it sends any message
> unsettled?  It expects to block until eventually it gets some
> indication of whether or not the message was delivered as intended.
> In the case of single consumer the expectation is obvious and well
> handled by the router.
>
> But in the case of multicast it is a different story: here we have the
> possibility that the message may be both consumed by one recipient and
> rejected by another.  So the question is: from the POV of the dev/app,
> what is the "obvious" default action the router should perform in that
> case?
>
> -K
>
>
> On Fri, Apr 13, 2018 at 3:38 PM, Chuck Rolke  wrote:
>> I would prefer to keep the feature enforced as it is now. I was one who
>> was surprised to have a sender whose message is settled by the router
>> only to find out that it was not delivered anywhere.
>>
>> The document 
>> https://qpid.apache.org/releases/qpid-dispatch-1.0.0/book/book.html#routing-patterns
>> needs to have a clearer explanation of the lossy nature of multicast
>> distribution.
>>
>> - Original Message -
>>> From: "Ted Ross" 
>>> To: users@qpid.apache.org
>>> Sent: Thursday, April 12, 2018 6:26:34 PM
>>> Subject: Re: Proposed Feature Removal from Dispatch Router
>>>
>>> For the record, here is the Jira for the feature in question:
>>>
>>> https://issues.apache.org/jira/browse/DISPATCH-744
>>>
>>> On Thu, Apr 12, 2018 at 6:20 PM, Ted Ross  wrote:
>>> > We added a feature back in 1.0.0 to reject unsettled deliveries to
>>> > multicast addresses by default.  This can be disabled through
>>> > configuration but is on by default.
>>> >
>>> > The rationale was that the router would accept and settle unsettled
>>> > multicasts even though it might not have delivered the messages to any
>>> > consumer.  The rejection with error code was intended to inform users
>>> > that they should pre-settle deliveries to multicast addresses in
>>> > keeping with the best-effort nature of multicast routing.
>>> >
>>> > In practice, this is more of an annoyance because none of the example
>>> > clients (and apparently the users' clients) actually do anything with
>>> > the error code in the rejected delivery.  The router appears to
>>> > silently drop such messages for no good reason and good will is wasted
>>> > in chasing down the issue to "oh, you should turn off this handy
>>> > feature".
>>> >
>>> > The recently raised https://issues.apache.org/jira/browse/DISPATCH-966
>>> > is caused by this feature as well.  This is because the router can
>>> > stream large messages in multiple transfers.  The first transfer is
>>> > used for routing and the last transfer should be used to determine the
>>> > settlement status of the delivery.  It is not a trivial fix to make
>>> > this work correctly.
>>> >
>>> > For the above two reasons, I propose that we back out this feature and
>>> > allow multicasting with unsettled deliveries.  We should add a clear
>>> > note in the documentation that states that multicast is best-effort,
>>> > regardless of the settlement status of the deliveries.
>>> >
>>> > Any objections from the users?
>>> >
>>> > -Ted
>>>
>>> -
>>> To unsubscribe, e-mail: 

Re: QDR: transacted connections with multiple brokers

[mlange]

Done: https://issues.apache.org/jira/browse/DISPATCH-968



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Qpid Dispatch authenticate through ldap, is this possible

[mlange]

> That looks a bit as if artemis is trying to authenticate the connection
> via a client certificate. From the config snippet you supplied it
> doesn't look like it is using TLS, let alone supplying a client cert.
> Are you able to get a protocol trace for the interaction between the
> router and the broker? (A simple way to do this would be to start a
> router with that connector in with env var PN_TRACE_FRM=1 and capture
> the output) 

It shouldn't do that, trying to authenticate via client certificate (well,
not yet at least)
With the same config, but then connecting directly to the broker (a
javax.jms.Connection(String user, String password); with the same
credentials) allows me to connect just fine.

The trace gives quite some output; I think the relevant parts are these:
[0x7f595400bdb0]:  -> SASL
[0x7f595400bdb0]:  <- SASL
[0x7f595400bdb0]:0 <- @sasl-mechanisms(64)
[sasl-server-mechanisms=@PN_SYMBOL[:PLAIN, :ANONYMOUS]]
[0x7f595400bdb0]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS,
initial-response=b"anonym...@masterbroker.host.name"]
[0x7f595400bdb0]:0 <- @sasl-outcome(68) [code=0]

Here it seems as if qpid chooses to use ANONYMOUS to connect with the broker
(which will not work, the broker is configured to require authentication)
whereas the broker seems to offer PLAIN as well.

a bit later I see the connection:
[0x7f5954027d60]:4 <- @begin(17) [next-outgoing-id=0,
incoming-window=2147483647, outgoing-window=2147483647]
[0x7f5954027d60]:4 <- @attach(18)
[name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c77f:1:1:1:testqueues.testqueue",
handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0,
source=@source(40) [address="ID:8b0bc583-315f-4f54-8f17-ecc40379c77f:1:1:1",
durable=0, timeout=0, dynamic=false,
outcomes=@PN_SYMBOL[:"amqp:accepted:list", :"amqp:rejected:list",
:"amqp:released:list", :"amqp:modified:list"]], target=@target(41)
[address="testqueues.testqueue", durable=0, timeout=0, dynamic=false,
capabilities=@PN_SYMBOL[:queue]], initial-delivery-count=0,
max-message-size=0]
[0x7f5954027d60]:4 -> @begin(17) [remote-channel=4, next-outgoing-id=0,
incoming-window=2147483647, outgoing-window=2147483647]
[0x7f595400bdb0]:0 -> @begin(17) [next-outgoing-id=0,
incoming-window=2147483647, outgoing-window=2147483647]
[0x7f595400bdb0]:0 -> @attach(18)
[name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c77f:1:1:1:testqueues.testqueue",
handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0,
source=@source(40) [address="ID:8b0bc583-315f-4f54-8f17-ecc40379c77f:1:1:1",
durable=0, timeout=0, dynamic=false,
outcomes=@PN_SYMBOL[:"amqp:accepted:list", :"amqp:rejected:list",
:"amqp:released:list", :"amqp:modified:list"]], target=@target(41)
[address="testqueues.testqueue", durable=0, timeout=0, dynamic=false,
capabilities=@PN_SYMBOL[:queue]], initial-delivery-count=0,
max-message-size=0]
[0x7f595400bdb0]:0 <- @close(24) [error=@error(29)
[condition=:"amqp:internal-error", description="Unrecoverable error:
AMQ119031: Unable to validate user from /192.168.0.1:52202. Username: null;
SSL certificate subject DN: unavailable"]]
[0x7f595400bdb0]:  <- EOS
[0x7f595400bdb0]:0 -> @close(24) []
[0x7f595400bdb0]:  -> EOS
[0x7f5954027d60]:4 -> @attach(18)
[name="qpid-jms:sender:ID:8b0bc583-315f-4f54-8f17-ecc40379c77f:1:1:1:testqueues.testqueue",
handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0,
source=@source(40) [durable=0, timeout=0, dynamic=false], target=@target(41)
[durable=0, timeout=0, dynamic=false], initial-delivery-count=0,
max-message-size=0]
[0x7f5954027d60]:4 -> @detach(22) [handle=0, closed=false, error=@error(29)
[condition=:"qd:routed-link-lost", description="Connectivity to the peer
container was lost"]]
[0x7f5954027d60]:4 <- @detach(22) [handle=0, closed=true]

Username is null, as well as client-certificates not provided (which is
logical, since there are none yet);

When I add saslMechanisms: PLAIN to the connection{} I see a new error in
the SERVER log module (server.log):
 proton:io:sasl_error SASL(-4): no mechanism available: No worthy mechs
found (Authentication failed [mech=none])

which is weird, as it seems that PLAIN is offered by the broker. (or I am
interpreting things completely wrong)



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Proposed Feature Removal from Dispatch Router

[Ken Giusti]

We really should try to do something smarter in the case of unsettled
multicast rather than either of the current approaches.

What does an application/dev expect when it sends any message
unsettled?  It expects to block until eventually it gets some
indication of whether or not the message was delivered as intended.
In the case of single consumer the expectation is obvious and well
handled by the router.

But in the case of multicast it is a different story: here we have the
possibility that the message may be both consumed by one recipient and
rejected by another.  So the question is: from the POV of the dev/app,
what is the "obvious" default action the router should perform in that
case?

-K


On Fri, Apr 13, 2018 at 3:38 PM, Chuck Rolke  wrote:
> I would prefer to keep the feature enforced as it is now. I was one who
> was surprised to have a sender whose message is settled by the router
> only to find out that it was not delivered anywhere.
>
> The document 
> https://qpid.apache.org/releases/qpid-dispatch-1.0.0/book/book.html#routing-patterns
> needs to have a clearer explanation of the lossy nature of multicast
> distribution.
>
> - Original Message -
>> From: "Ted Ross" 
>> To: users@qpid.apache.org
>> Sent: Thursday, April 12, 2018 6:26:34 PM
>> Subject: Re: Proposed Feature Removal from Dispatch Router
>>
>> For the record, here is the Jira for the feature in question:
>>
>> https://issues.apache.org/jira/browse/DISPATCH-744
>>
>> On Thu, Apr 12, 2018 at 6:20 PM, Ted Ross  wrote:
>> > We added a feature back in 1.0.0 to reject unsettled deliveries to
>> > multicast addresses by default.  This can be disabled through
>> > configuration but is on by default.
>> >
>> > The rationale was that the router would accept and settle unsettled
>> > multicasts even though it might not have delivered the messages to any
>> > consumer.  The rejection with error code was intended to inform users
>> > that they should pre-settle deliveries to multicast addresses in
>> > keeping with the best-effort nature of multicast routing.
>> >
>> > In practice, this is more of an annoyance because none of the example
>> > clients (and apparently the users' clients) actually do anything with
>> > the error code in the rejected delivery.  The router appears to
>> > silently drop such messages for no good reason and good will is wasted
>> > in chasing down the issue to "oh, you should turn off this handy
>> > feature".
>> >
>> > The recently raised https://issues.apache.org/jira/browse/DISPATCH-966
>> > is caused by this feature as well.  This is because the router can
>> > stream large messages in multiple transfers.  The first transfer is
>> > used for routing and the last transfer should be used to determine the
>> > settlement status of the delivery.  It is not a trivial fix to make
>> > this work correctly.
>> >
>> > For the above two reasons, I propose that we back out this feature and
>> > allow multicasting with unsettled deliveries.  We should add a clear
>> > note in the documentation that states that multicast is best-effort,
>> > regardless of the settlement status of the deliveries.
>> >
>> > Any objections from the users?
>> >
>> > -Ted
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
>> For additional commands, e-mail: users-h...@qpid.apache.org
>>
>>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>



-- 
-K

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: QDR: transacted connections with multiple brokers

[Alan Conway]

On Mon, Apr 16, 2018 at 9:10 AM, mlange  wrote:

> aconway.rh wrote
> > On Mon, Apr 16, 2018 at 8:52 AM, Gordon Sim 
>
> > gsim@
>
> >  wrote:
> >
> >
> >>
> >> No. The problem is that a transaction could involve messages to/from
> >> different brokers, i.e. the transaction would become a distributed
> >> transaction. At present there is no support for that in the router.
> >
> >
> > We talked before about this: the idea was to start a non-distributed TX
> > based on the first broker involved, and abort it if it attempts to spread
> > to more brokers. However this approach would really create a lot of
> > potential for error and confusion, as well as complexity in the router.
> > The
> > conclusion was that such a partial solution would probably do more harm
> > than good, and if we want to move towards multi-broker TX we should put
> > our
> > effort into proper standard distributed TX and not some halfway solution.
> >
> >
> >>
> >> -
> >> To unsubscribe, e-mail:
>
> > users-unsubscribe@.apache
>
> >> For additional commands, e-mail:
>
> > users-help@.apache
>
> >>
> >>
>
> Agree with that one; a proper distributed TX would indeed be the best
> solution; Couldn't find any issue in the Jira, are there plans to build
> this? I think it would be a great addition.
>

No immediate plans that I'm aware of. It has always been a potential
feature but never at the top of the priority pile. By all means raise it,
so we can track interest and maybe encourage someone to contribute some
code.

Re: QDR: transacted connections with multiple brokers

[mlange]

aconway.rh wrote
> On Mon, Apr 16, 2018 at 8:52 AM, Gordon Sim 

> gsim@

>  wrote:
> 
> 
>>
>> No. The problem is that a transaction could involve messages to/from
>> different brokers, i.e. the transaction would become a distributed
>> transaction. At present there is no support for that in the router.
> 
> 
> We talked before about this: the idea was to start a non-distributed TX
> based on the first broker involved, and abort it if it attempts to spread
> to more brokers. However this approach would really create a lot of
> potential for error and confusion, as well as complexity in the router.
> The
> conclusion was that such a partial solution would probably do more harm
> than good, and if we want to move towards multi-broker TX we should put
> our
> effort into proper standard distributed TX and not some halfway solution.
> 
> 
>>
>> -
>> To unsubscribe, e-mail: 

> users-unsubscribe@.apache

>> For additional commands, e-mail: 

> users-help@.apache

>>
>>

Agree with that one; a proper distributed TX would indeed be the best
solution; Couldn't find any issue in the Jira, are there plans to build
this? I think it would be a great addition.



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: QDR: transacted connections with multiple brokers

[Alan Conway]

On Mon, Apr 16, 2018 at 8:52 AM, Gordon Sim  wrote:

> On 16/04/18 12:30, mlange wrote:
>
>> Hi all,
>>
>> QDR has some limited support for transactions, seeing a special endpoint
>> $coordinator.
>> When one has QDR with only one broker behind it you can do a
>>
>> linkRoute {
>>prefix: $coordinator
>>dir: in
>>conn: broker1
>> }
>>
>> and the same, but then dir: out
>>
>> However, this will work for one broker. But given the situation when one
>> has
>> multiple brokers defined in one QDR, how can this be solved? Is there
>> something after $coordinator that can be used as prefix instead? Can QDR
>> tell which broker should get which transaction messages based on the
>> endpoint and to which broker those messages get linkRouted?
>>
>
> No. The problem is that a transaction could involve messages to/from
> different brokers, i.e. the transaction would become a distributed
> transaction. At present there is no support for that in the router.


We talked before about this: the idea was to start a non-distributed TX
based on the first broker involved, and abort it if it attempts to spread
to more brokers. However this approach would really create a lot of
potential for error and confusion, as well as complexity in the router. The
conclusion was that such a partial solution would probably do more harm
than good, and if we want to move towards multi-broker TX we should put our
effort into proper standard distributed TX and not some halfway solution.


>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: QDR: transacted connections with multiple brokers

[Gordon Sim]

On 16/04/18 12:30, mlange wrote:

Hi all,

QDR has some limited support for transactions, seeing a special endpoint
$coordinator.
When one has QDR with only one broker behind it you can do a

linkRoute {
   prefix: $coordinator
   dir: in
   conn: broker1
}

and the same, but then dir: out

However, this will work for one broker. But given the situation when one has
multiple brokers defined in one QDR, how can this be solved? Is there
something after $coordinator that can be used as prefix instead? Can QDR
tell which broker should get which transaction messages based on the
endpoint and to which broker those messages get linkRouted?


No. The problem is that a transaction could involve messages to/from 
different brokers, i.e. the transaction would become a distributed 
transaction. At present there is no support for that in the router.


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Qpid Dispatch authenticate through ldap, is this possible

[Gordon Sim]

On 16/04/18 12:21, mlange wrote:

But then I looked in the broker log... and there it is:
2018-04-16 13:05:04,657 WARN
[org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler]
AMQ119031: Unable to validate user from /192.168.0.1:33034. Username: null;
SSL certificate subject DN: unavailable:
ActiveMQAMQPInternalErrorException[errorType=INTERNAL_ERROR
message=AMQ119031: Unable to validate user from /192.168.0.1:33034.
Username: null; SSL certificate subject DN: unavailable]


That looks a bit as if artemis is trying to authenticate the connection 
via a client certificate. From the config snippet you supplied it 
doesn't look like it is using TLS, let alone supplying a client cert. 
Are you able to get a protocol trace for the interaction between the 
router and the broker? (A simple way to do this would be to start a 
router with that connector in with env var PN_TRACE_FRM=1 and capture 
the output)


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

QDR: transacted connections with multiple brokers

[mlange]

Hi all,

QDR has some limited support for transactions, seeing a special endpoint
$coordinator.
When one has QDR with only one broker behind it you can do a 

linkRoute {
  prefix: $coordinator
  dir: in
  conn: broker1
}

and the same, but then dir: out

However, this will work for one broker. But given the situation when one has
multiple brokers defined in one QDR, how can this be solved? Is there
something after $coordinator that can be used as prefix instead? Can QDR
tell which broker should get which transaction messages based on the
endpoint and to which broker those messages get linkRouted?




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Qpid Dispatch authenticate through ldap, is this possible

[mlange]

So, now I have QDR able to authenticate via LDAP via SASL; When running some
tests, I got some errors that I could not relate... MessageProducer is
closed as IllegalStateException; Didn't have a clue...

But then I looked in the broker log... and there it is:
2018-04-16 13:05:04,657 WARN 
[org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler]
AMQ119031: Unable to validate user from /192.168.0.1:33034. Username: null;
SSL certificate subject DN: unavailable:
ActiveMQAMQPInternalErrorException[errorType=INTERNAL_ERROR
message=AMQ119031: Unable to validate user from /192.168.0.1:33034.
Username: null; SSL certificate subject DN: unavailable]

The connector to the broker is built up like this:
connector {
name: broker1-conn
role: route-container
host: masterbroker.host.name
port: 10010
failoverList: amqp://slavebroker.host.name:10010
#   saslMechanisms: DIGEST-MD5 PLAIN LOGIN EXTERNAL 
saslUsername: 
saslPassword: 
}

Apache Artemis does not use SASL, could that be the reason it does not see
any username provided?
In the java code I create a session with user X, QDR on the listener side
connects to LDAP as user Y (which has proxy authentication enabled as
authzTo), which works. and then I have this connector which actually is the
same as user X.

How do I make a valid authenticated connection as QDR to my Artemis broker?



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Auth Service in Dispatch Router 1.0.1

[Gordon Sim]

On 09/04/18 18:16, Hudalla Kai (INST/ECS4) wrote:

Hi,

I have implemented an auth service based on the "spec" given in [1].
When I configure our dispatch router 1.0.1 to use the service, I can see in the
log file of the auth service that the router indeed delegates authentication to
the auth service but doesn't seem to include the "ADDRESS_AUTHZ" desired
capability in its open frame so that the service doesn't recognize that it 
should
reply accordingly by including the client's permissions in the open frame's
"address-authz" property. In fact, the router doesn't seem to send any desired
capability at all AFAICT.


That was added after the 1.0.0 branch; it is on master hwoever and will 
be in the next (1.1.0) release).


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: [Java Broker] Temporary queues ACLs for multiple users

[Vavricka]

Hi Keith,

excellent idea, this will completely solve our use case.

Regards,
Tomas



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org