I made a simpler test, without using tulip, just using plain
sockets<http://stackoverflow.com/questions/21973661/os-x-udp-send-error-55-no-buffer-space-available/21973705?noredirect=1#comment33297277_21973705>
.
from socket import *
udp = socket(AF_INET, SOCK_DGRAM)
udp.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)
udp.bind(('0.0.0.0', 1337))
udp.setblocking(False)
udp.setsockopt(SOL_IP, IP_TTL, 4)
udp.connect(('8.8.8.8', 12345))
buf = b'x' * 400for _ in range(1024 * 1024 * 10):
udp.send(buf)
In this test I'm just writing a lot of udp packets to 8.8.8.8, which gets
dropped after 4 hops (it's just for testing anyway). But this code actually
causes the same error (so it has nothing to do with tulip, though
flow-control might be affected).
The weird thing is, that the following code although causes the same error:
from socket import *
udp = socket(AF_INET, SOCK_DGRAM)
udp.connect(('8.8.8.8', 12345))
buf = b'x' * 400for _ in range(1024 * 1024 * 10):
udp.send(buf)
In other words: The "blocking" mode is not so blocking apparently on my
machine. If you cannot reproduce this error(which is good!), than this
might be an issue on my local machine. But in any case it's not a tulip bug.
I'm just curious what this can be...
Am Sonntag, 23. Februar 2014 23:34:39 UTC+1 schrieb Guido van Rossum:
>
> I haven't been able to repro this using the test you attached to issue
> 153. But I don't have another machine available, I've only tried localhost
> on OSX (10.9) and from that same box to a local virtual machine running
> Ubunu. The receiving end just throws away the data -- is that your test
> setup too? I used "while 1: s.recv(10000); n += 1" in an interactive Python
> shell.
>
> I do see packet loss (just counting packets received and packets sent) but
> no exceptions nor does pause_writing() ever get called. From putting some
> print()s in the code it looks like the sendto() operation always
> immediately succeeds.
>
> The errno has a name: errno.ENOBUFS -- it's 55 on OSX, but 105 on Ubuntu.
>
> Perhaps you can elaborate on your test setup?
>
>
> On Sun, Feb 23, 2014 at 11:29 AM, Christopher Probst <
> [email protected] <javascript:>> wrote:
>
>> Ok, I tried it now with 180 bytes packet and it does not occur, I guess
>> that my lan connection(gigabit) is too fast that those packets get queued
>> up. Or OSX decide whether or not to queue those packets based on the size.
>> But this seems to be heavily os dependent .
>>
>> Am Sonntag, 23. Februar 2014 20:12:46 UTC+1 schrieb Guido van Rossum:
>>>
>>> Have you tried reducing the write buffer size?
>>> On Feb 23, 2014 10:41 AM, "Christopher Probst" <foxnet.d...@googlemail.
>>> com> wrote:
>>>
>>>> I saw this in the official 3.4rc1 doc.
>>>>
>>>> The doc says, that flow-control callbacks are valid for Protocols and
>>>> SubprocessProtocols, but DatagramProtocol is not specified.
>>>>
>>>> Though I'm happy that Datagram control flow is officially supported I
>>>> have now an other problem which is directly connected with this issue.
>>>> Using OSX10.9.1 and python 3.3 sending a lot of udp packets actually
>>>> does not cause the flow-control to be activated but throws an OSError(55
>>>> *No
>>>> buffer space available)* .
>>>>
>>>> After googling around for hours I found out that this is a BSD(probably
>>>> OS X only) thing. On linux the control-flow works as expected.
>>>>
>>>> I listed an issue for this in your tulip repo:
>>>> Issue 153 <https://code.google.com/p/tulip/issues/detail?id=153>
>>>>
>>>> I think that this is really not a python bug, but the way BSD/OSX
>>>> handles too much udp packets (the socket sdnbuf option is kind of ignored
>>>> by OSX). I've read that windows actually handles udp overload in a
>>>> similiar
>>>> way as BSD does. Though i don't have a windows machine this should
>>>> probably
>>>> be tested to confirm or disprove this issue.
>>>>
>>>> PS:
>>>> 18.5.3.2.5. Flow control
>>>> callbacks<http://docs.python.org/3.4/library/asyncio-protocol.html#flow-control-callbacks>
>>>>
>>>>
>>>> These callbacks may be called on
>>>> Protocol<http://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.Protocol>
>>>> and
>>>> SubprocessProtocol<http://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.SubprocessProtocol>
>>>> instances:
>>>> BaseProtocol.pause_writing()<http://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseProtocol.pause_writing>
>>>>
>>>>
>>>> Called when the transport’s buffer goes over the high-water mark.
>>>> BaseProtocol.resume_writing()<http://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseProtocol.resume_writing>
>>>>
>>>>
>>>> Called when the transport’s buffer drains below the low-water mark.
>>>>
>>>>
>>>>
>>>>
>>>> Am Sonntag, 23. Februar 2014 18:56:42 UTC+1 schrieb Guido van Rossum:
>>>>>
>>>>> This looks like a bug in the docs; the intention is that datagram
>>>>> protocols also support flow control. Where does it say so in the docs? Is
>>>>> it the PEP or the CPython Doc tree?
>>>>>
>>>>>
>>>>> On Sat, Feb 22, 2014 at 5:12 PM, Christopher Probst <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> after looking into the implementation I saw that, for instance, _
>>>>>> SelectorDatagramTransport calls _maybe_pause_protocol and it's
>>>>>> counterpart, but the doc says that only Protocol and SubprocessProtocol
>>>>>> has
>>>>>> flow-control support and DatagramProtocol does not.
>>>>>>
>>>>>> I know that udp flow-control is not the same as tcp flow-control, but
>>>>>> I'm concerned about filling up the internal buffer when writing a lot of
>>>>>> datagrams. If this is not supported, I would argue that the udp
>>>>>> support is pretty much broken for data intense application because how
>>>>>> would the writer know when the internal buffer (and/or kernel level
>>>>>> buffer)
>>>>>> are full ?
>>>>>>
>>>>>> So, is the doc just not up-to-date or is it an implementation detail
>>>>>> of tulip ?
>>>>>>
>>>>>> And other question that came up: Are there any plans for coroutine
>>>>>> methods for udp (like StreamWriter/Reader for TCP) ?
>>>>>>
>>>>>> Also, are there any "dirty" corners somebody heavily working with udp
>>>>>> have to know ? I'm implementing reliable udp and I would like to use the
>>>>>> coroutine style instead of callbacks.
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Chris
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --Guido van Rossum (python.org/~guido)
>>>>>
>>>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
