I am trying to extract bulk messages from azure SB.
As per their documentation the azure SDK doesn't support bulk read message
and recommends using the native amqp for the Azure Service Bus. White
trying to negotiate a session with the azure SB, i noticed that independent
of what client is requesting, the SB dials down the incoming_window=5000.
This limits a max of 5000B send per bulk read, thus my consumer runs dry
till the RTT (which is large for inter cloud) to fetch a new packet.
Is this a restriction of the azure SB or am i not setting any/some of the
parameters correctly from the client side to achieve the negotiated window
size > 5000B.
I am using python proton MessagingHandler Class and clearly see that
on_message is called for few pkts composing of buffer size ~5000B and then
have to wait for the RTT delay to get the next batch.
Any suggestion to work around this problem and get larger bulked message ?
I can't reduce the RTT between the server and consumer. have some
workaround for parallel consumers but would like to solve the bulk problem
as that is most efficient way of achieving the high throughput.
class Recv(MessagingHandler):
def __init__(self):
super(Recv, self).__init__(prefetch=100, auto_accept=True,
auto_settle=True, peer_close_is_error=False)
def on_start(self, event):
conn = event.container.connect(connString)
event.container.create_receiver(conn, subscription)
def on_message(self, event):
print(event.message.body)
print datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3],
self.count, event.receiver.queued
[0xace380]: -> SASL
[0xace380]: <- SASL
[0xace380]:0 <- @sasl-mechanisms(64)
[sasl-server-mechanisms=@PN_SYMBOL[:MSSBCBS, :PLAIN, :ANONYMOUS, :EXTERNAL]]
[0xace380]:0 -> @sasl-init(65) [mechanism=:PLAIN,
initial-response=b"\x00iothubroutes_XXXXX\XXXXX="]
[0xace380]:0 <- @sasl-outcome(68) [code=0, additional-data=b"Welcome!"]
[0xace380]: -> AMQP
[0xace380]:0 -> @open(16)
[container-id="0ad171ca-cefa-4a27-a7dc-0520e5393fa5", hostname="
nebhubsb.servicebus.windows.net", channel-max=32767]
[0xace380]:0 -> @begin(17) [next-outgoing-id=0, incoming-window=2147483647,
outgoing-window=2147483647]
[0xace380]:0 -> @attach(18)
[name="0ad171ca-cefa-4a27-a7dc-0520e5393fa5-kukatopic/Subscriptions/kukasub",
handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0,
source=@source(40) [address="kukatopic/Subscriptions/kukasub", durable=0,
timeout=0, dynamic=false], target=@target(41) [durable=0, timeout=0,
dynamic=false], initial-delivery-count=0, max-message-size=0]
[0xace380]:0 -> @flow(19) [incoming-window=2147483647, next-outgoing-id=0,
outgoing-window=2147483647, handle=0, delivery-count=0, link-credit=100,
drain=false]
[0xace380]: <- AMQP
[0xace380]:0 <- @open(16)
[container-id="b970f07881334c658eb80ff336f2a683_G16", max-frame-size=65536,
channel-max=4999, idle-time-out=240000]
[0xace380]:0 <- @begin(17) [remote-channel=0, next-outgoing-id=1,
incoming-window=5000, outgoing-window=2147483647, handle-max=255]
[0xace380]:0 <- @attach(18)
[name="0ad171ca-cefa-4a27-a7dc-0520e5393fa5-kukatopic/Subscriptions/kukasub",
handle=0, role=false, rcv-settle-mode=1, source=@source(40)
[address="topic/Subscriptions/sub", durable=0, timeout=0, dynamic=false],
target=@target(41) [durable=0, timeout=0, dynamic=false],
initial-delivery-count=0, max-message-size=266240]
