Re: [prosody-dev] proxy65: buffer size too small ?
Le samedi 21 novembre 2015, 00:04:04 Matthew Wild a écrit : > On 20 November 2015 at 23:42, Goffi wrote: > > Le vendredi 20 novembre 2015, 23:22:54 Matthew Wild a écrit : > >> On 20 November 2015 at 23:15, Goffi wrote: > >> > >> What version of Prosody are you using? > > > > current trunk tip (3c4799467dae) > > > > I have the feeling that with a 64Kio chunk size, I sending data too > > quickly > > for gajim to read, and after a while the prosody buffer explode. > > Correct. The error that gets logged indicates that the receiving > client was not receiving fast enough. However the proxy65 code is > specifically designed so that can't happen. I'll see if I can > reproduce it. If it's not much trouble, could you try with libevent > and see if that changes anything? Yes I will, but maybe not tonight, it's getting late :) Good night, and thanks for your help. -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at http://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] proxy65: buffer size too small ?
On 20 November 2015 at 23:42, Goffi wrote: > Le vendredi 20 novembre 2015, 23:22:54 Matthew Wild a écrit : >> On 20 November 2015 at 23:15, Goffi wrote: > >> What version of Prosody are you using? > > current trunk tip (3c4799467dae) > > I have the feeling that with a 64Kio chunk size, I sending data too quickly > for gajim to read, and after a while the prosody buffer explode. Correct. The error that gets logged indicates that the receiving client was not receiving fast enough. However the proxy65 code is specifically designed so that can't happen. I'll see if I can reproduce it. If it's not much trouble, could you try with libevent and see if that changes anything? Regards, Matthew -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at http://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] proxy65: buffer size too small ?
Le vendredi 20 novembre 2015, 23:22:54 Matthew Wild a écrit : > On 20 November 2015 at 23:15, Goffi wrote: > What version of Prosody are you using? current trunk tip (3c4799467dae) I have the feeling that with a 64Kio chunk size, I sending data too quickly for gajim to read, and after a while the prosody buffer explode. I'm currenlty having a similar error with http_upload: if I try to send a very big file (~ 400 Mio), I'm disconnected. I know that http_upload is more here to transfer small images, I'm just stressing it up a bit. -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at http://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] proxy65: buffer size too small ?
On 20 November 2015 at 23:15, Goffi wrote: > Hi Matthew, > > sorry for the late answer, I was quite busy. > > So it's not working neither with the network_default_read_size option you gave > me, and I'm not using libevent. What version of Prosody are you using? Regards, Matthew -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at http://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] proxy65: buffer size too small ?
Hi Matthew, sorry for the late answer, I was quite busy. So it's not working neither with the network_default_read_size option you gave me, and I'm not using libevent. For the context, it's a SI (XEP-0096) file transfer, I'm sending the file (~400 Mio) and I'm forcing the use of the proxy for the test. My chunks have a size of 2**16 (64 Kio) when disconnection occurs. If I set a size of 4096, there is no problem. I'm sending the file, so Gajim is actually not in cause here. If I do a direct connection I have no problem neither. Here is the significant part of the logs: --- logs socket debugserver.lua: accepted new client connection from 127.0.0.1:42662 to 5000 mod_proxy65debugSOCKS5 greeting complete mod_proxy65debugSOCKS5 target connected for session e83c1d0c64441f3a22a5f74a27d4f7dd17a8c066 c2s1b3e530 debugReceived[c2s]: necton3.int:log_events debugnecton3.int firing event: pre- iq/full necton3.int:log_events debugnecton3.int firing event: iq/full socket debugserver.lua: accepted new client connection from 127.0.0.1:42664 to 5000 mod_proxy65debugSOCKS5 greeting complete mod_proxy65debugSOCKS5 initiator connected for session e83c1d0c64441f3a22a5f74a27d4f7dd17a8c066 c2s1c0bac0 debugReceived[c2s]: necton3.int:log_events debugnecton3.int firing event: pre- iq/host proxy.necton3.int:proxy65 debugTransfer activated (sid: 5620a228-2256-4004-9149-6ca8593a461b, initiator: go...@necton3.int/SàT, target: lou...@necton3.int/Gajim) mod_proxy65debugondisconnect: 'receive buffer exceeded' mod_proxy65debugondisconnect: 'false' socket debugserver.lua: closed client handler and removed socket from list socket debugserver.lua: closed client handler and removed socket from list --- end logs I have added myself the "ondisconnect" log, because else mod_proxy65 was not logging the err. The xml stream is not disconnected, it's the Socks5 client only which is disconnected. So I currently workaround this by setting a 4Kio limit for proxies, which is a reasonable value anyway. Hope this help Cheers Goffi Le mercredi 11 novembre 2015, 22:51:23 Matthew Wild a écrit : > Hi, > > On 11 November 2015 at 21:19, Goffi wrote: > > G'day, > > > > I'm currently working on file transfer, and I realized that default > > minimum > > buffer size for proxy65 is 4096 octets. If a client send a bigger one, it > > will be disconnected. > > That's not intentional (obviously). Filling the buffer in mod_proxy65 > is harmless and not an error. > > I suspect you may be encountering an unrelated known issue. Are you > using libevent with Prosody? Does the client get disconnected, or does > the connection freeze? > > Try setting the following in your config file (global section) and see > if the issue goes away: > > network_default_read_size = "*a" > > If that doesn't fix it, please supply debug logs. > > > I haven't found documentation about a recommended buffer size, but 4Kio > > seems really low to me, especially if going above disconnect the client. > > 64 Kio seems more usual. In fact I have tried with Gajim, and it is > > disconnected when sending a file through the proxy. > > On my client I have set the size to 4Kio because of this, but I wonder if > > the subject has already been discussed here (after all, I guess I'm not > > the first one to send a file through prosody proxy with gajim). > > So if you have any hint on why this limit has been choosed, and if I need > > to report that to Gajim developers, i would appreciate. > > If Prosody is disconnecting the client here, it's most likely a bug on > our side, not Gajim. And you're certainly not the first person to send > a file through mod_proxy65, in the past I have stress-tested it, > streaming many GB through at a good speed. The buffer is a part of > that, and I think 4KB is fine for general use and public servers. It > seems it's not configurable however, and we should probably fix > that... > > Regards, > Matthew -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at http://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] proxy65: buffer size too small ?
Hi, On 11 November 2015 at 21:19, Goffi wrote: > G'day, > > I'm currently working on file transfer, and I realized that default minimum > buffer size for proxy65 is 4096 octets. If a client send a bigger one, it will > be disconnected. That's not intentional (obviously). Filling the buffer in mod_proxy65 is harmless and not an error. I suspect you may be encountering an unrelated known issue. Are you using libevent with Prosody? Does the client get disconnected, or does the connection freeze? Try setting the following in your config file (global section) and see if the issue goes away: network_default_read_size = "*a" If that doesn't fix it, please supply debug logs. > I haven't found documentation about a recommended buffer size, but 4Kio seems > really low to me, especially if going above disconnect the client. > 64 Kio seems more usual. In fact I have tried with Gajim, and it is > disconnected when sending a file through the proxy. > On my client I have set the size to 4Kio because of this, but I wonder if the > subject has already been discussed here (after all, I guess I'm not the first > one to send a file through prosody proxy with gajim). > So if you have any hint on why this limit has been choosed, and if I need to > report that to Gajim developers, i would appreciate. If Prosody is disconnecting the client here, it's most likely a bug on our side, not Gajim. And you're certainly not the first person to send a file through mod_proxy65, in the past I have stress-tested it, streaming many GB through at a good speed. The buffer is a part of that, and I think 4KB is fine for general use and public servers. It seems it's not configurable however, and we should probably fix that... Regards, Matthew -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at http://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Re: [prosody-dev] proxy65: buffer size too small ?
Le mercredi 11 novembre 2015, 22:19:26 Goffi a écrit : > G'day, > > I'm currently working on file transfer, and I realized that default minimum > buffer size for proxy65 is 4096 octets. If a client send a bigger one, it > will be disconnected. > > I haven't found documentation about a recommended buffer size, but 4Kio > seems really low to me, especially if going above disconnect the client. 64 > Kio seems more usual. In fact I have tried with Gajim, and it is > disconnected when sending a file through the proxy. > On my client I have set the size to 4Kio because of this, but I wonder if > the subject has already been discussed here (after all, I guess I'm not the > first one to send a file through prosody proxy with gajim). > So if you have any hint on why this limit has been choosed, and if I need to > report that to Gajim developers, i would appreciate. > > Thanks ! > Goffi I was of course talking about the default *maximum* buffer size ! -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to prosody-dev@googlegroups.com. Visit this group at http://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
