On 02/24/2014 12:51 PM, Tor Rune Skoglund wrote:
Hello list,
we are setting up an AMQP messaging infrastructure using Qpid (currently
using proton and the dispatch router, not the broker). This is the basic
set-up:
* We have a server side, with a public, known IP-address and port.
* We have X number of clients going on/off the net at unpredictable
times and intervals. These clients may be NAT'ed (on possible separate
LANs), and might change public and local IP-address at random.
* All (server and clients) are running the dispatch router locally, and
connect to the server's dispatch router address on startup
automatically. (The server's address is currently hard-coded in the
clients' dispatch routers' configuration)
We have successful in getting clients and server communicate all with
each other using the server's public address/port as the only known
connection point in the system. All in all, things look very promising! :-)
Now we are to move on to next steps. There are especially these features
we need to make sure we support:
a) Sending a message _from_ a device that is not presently on-line at
the moment:
Devices come on/off the net at unpredictable times and intervals.
However, there might be messages _from_ the device that needs to be
delivered to the server when it comes on. On the application level on a
client, we just want to deliver a message to the local "message handler"
(i.e. the dispatch router presently), and let the application program
continue working without halt. For this functionality, is it right to
assume that we need to implement the broker also, locally, as this is
not a dispatch router feature?
You need something to periodically retry the connection and when
successful send any queued messages. As you say dispatch router does not
provide this.
The aim is for the proton messenger client to provide this sort of
capability at some point in the future, but it isn't there yet I'm afraid.
You could use a broker co-located with your sender, provided it has the
ability to establish outgoing reconnections over which it can deliver
the messages when connected.
Alternatively you would at present need to build the logic yourself. You
could do this as a specialised intermediary perhaps (rather than in the
sending code itself). I.e. you would be building a very simple
'broker'like thing, more specialised to this one role. This would be
colocated with your sender and would do the storing of messages (plus
any expiration needed), when they could not be directly forwarded to the
gateway.
b) Sending a message _to_ a device that is not present on-line at the
moment:
In case the device is not presently on-line, there will be no active
route to it. However, it should still be addressable (aka. a
"registered" device in the system), and messages for it (from any other
device - server or other clients) should be delivered when it is (might
be) back on-line. This relates to the previous point, but as far as we
can see, for this to work, the router would need to know that the
address for the device is still "legal" even though not on-line, and be
able to deliver the message when it actually comes back on-line. As seen
from the application program, the situation is the same as the previous
point. Is it right to assume that we broker for this?
Yes, this is essentially 'store-and-forward' behaviour, which the router
does not itself provide. One question, do senders generally care about
if/when the actual receiver gets the message? or are they happy as long
as it has been accepted into some queue pending delivery to the receiver
when it comes back on line? Or perhaps the acknowledgement of messages
is not that important but you would like to minimise loss?
c) Time to live on messages:
Every message should have a time-to-live parameter. I.e., if the message
is not delivered before ttl is expired (ref above cases), the message
can be discarded at the local "message handler" level. Is this AMQP/Qpid
functionality; which components do we need to support it?
Both broker implementations support it, so if you do use a broker queue
to store messages pending delivery then the ttl provides one means to
limit the growth of that queue (which is a concern in both the send and
receive case if the client is disconnected for a 'long' time, where
'long' really depends on the size and rate of sending).
d) Resuming large messages:
The payload of messages could be up to (in really worst cases) several
gigabytes. Is there AMQP/Qpid functionality at some level that makes the
transfer resume (rather than restart) in case the is a network dropout
during transfer? Where does this functionality exist?
Broker/router/messager lib...?
The AMQP protocol supports this, but it isn't supported in the c++
broker, the dispatch router or I don't think the messenger client itself.
Can the qpid broker replace all dispatch router functionality in these
cases, or is it an addition to the dispatch router functionality?
The qpid router provides end-to-end acknowledgement, i.e. the sender
gets an acknowledgement from the client, via the router, whereas a
broker will give the sender an acknowledgement independently of
delivering the message to a consumer.
The qpid router supports redundant routes between router instances,
which helps building fault tolerance. The c++ brokers federation does
not do this.
Other than that you could replace the router with a broker if needed.
However there is some work planned for the dispatch router that should
allow it to work well alongside brokers. The implementation is not quite
there yet however.
I hope this helps a little, your project sounds very interesting and I'm
sure there will be many of the list willing to help and offer
opinions/advice, so please don't hesitate to ask any further questions etc!
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
