Re: Connection Resetting
A little more about how my proxy works, My proxy behaves like number two in the previous email. It is purely a bidirectional proxy with no clear tracking of the protocol. With the only exception being it reads the entire (unencrypted) Connect request from the client, and then goes and connects to the server. Once the connection to the server is OK it creates two separate structures that are wrappers for communication over the ssl protocol. These structures are both aware of each other, and can schedule a write, or a read with the other. It's multi threaded with non-blocking I/O. I'm not sure exactly what you mean by socket discovery, but I think you are asking how my program determines when something is ready? If that's the case then my program uses a select statement to watch the file descriptor to see if it's ready for read or write. It uses a call back system to perform the correct action based on which fd_set was ready. The issue with my file upload shouldn't be lying in the blocking issue since it's non-blocking I/O. It looks like the issue with a SSL_read call from the perspective of the program as the client is returning zero during the upload (which makes sense) but the way my proxy is handling connections the only reason it could have read is if there was a SSL_get_error() that returned a SSL_ERROR_WANT_READ code. I am going to post my ssl_read and ssl_write wrapper functions, and anything else that's important to those two here: void ProxySSLConnection::ssl_read() { if (!_ssl) return; DataBuffer *buf = _other-_buffer; int ret = SSL_read(_ssl, buf-end(), buf-len_avail()); if (ret 0) { buf-len += ret; _other-schedule_send_buffer(); schedule_read(ProxySSLConnection::ssl_read); } else { handle_ssl_error(ret, ProxySSLConnection::ssl_read, read); } } void ProxySSLConnection::ssl_write() { if (!_ssl) return; int ret = SSL_write(_ssl, _buffer-begin(), _buffer-len); //printf(WRITING: \n); //These lines are just debugging. //for(char * buf = _buffer-begin();buf != _buffer-end();buf++){ // printf(%c, *buf); //} if (ret 0) { _buffer-erase_front(ret); if (!_buffer-empty()) { schedule_write(ProxySSLConnection::ssl_write); } } else { handle_ssl_error(ret, ProxySSLConnection::ssl_write, write); } } The handle_ssl_error function is: void ProxySSLConnection::handle_ssl_error(int ret, handler_function handler, const char * caller) { int error = SSL_get_error(_ssl, ret); switch (error) { case SSL_ERROR_WANT_READ: schedule_read(handler); break; case SSL_ERROR_WANT_WRITE: schedule_write(handler); break; case SSL_ERROR_ZERO_RETURN: _proxy_thread-shutdown(); break; default: unsigned long err = ERR_get_error(); while (err != 0) { char* err_string = ERR_error_string(err, NULL); cout err_string endl; err = ERR_get_error(); } close_connection(); _proxy_thread-shutdown(); break; } } If the following case is added to the switch then the upload is fixed, but the threads never exit: case SSL_ERROR_SYSCALL: //This case is where the uploads fail. schedule_write(handler); break; The caller variable is just for debug purposes. As per Dave Thompson's request I was using it to determine which call was causing the error, and was then calling perror(caller) since I was getting a SSL_ERROR_SYSCALL with a ret of 0, and the perror printed read: Success. Schedule Read and Write: void ProxySSLConnection::schedule_write(handler_function handler) { _write_handler = handler; _proxy_thread-register_write_callback(this); } void ProxySSLConnection::schedule_read(handler_function handler) { _read_handler = handler; _proxy_thread-register_read_callback(this); } _proxy_thread is the calling thread that created the ProxySSLConnection class. register_read and register_write callbacks are literally just doing an FD_SET() on the file descriptor of the _ssl connection and a fd_set called read_set, and write_set respectively. In each ProxyThread there select statement inside of a main loop that is woken up by either a signal from another thread, or from a file descriptor being ready. When the file descriptor is ready the thread will call on_read or on_write of the appropriate ProxySSLConnection on_read, and on_write: void ProxySSLConnection::on_read() { handle_callback(_read_handler); } void ProxySSLConnection::on_write() { handle_callback(_write_handler); } void ProxySSLConnection::handle_callback(handler_function *handler) { if (*handler) { handler_function the_handler = *handler;
RE: Connection Resetting
From: owner-openssl-us...@openssl.org On Behalf Of Sam Jantz Sent: Wednesday, 01 September, 2010 12:25 My proxy behaves like number two in the previous email. It is purely a bidirectional proxy with no clear tracking of the protocol [after CONNECT] It's multi threaded with non-blocking I/O. I'm not sure exactly what you mean by socket discovery, but I think you are asking how my program determines when something is ready? If that's the case then my program uses a select statement to watch the file descriptor to see if it's ready for read or write [with callbacks] based on which fd_set was ready. Okay, sounds fine. It looks like the issue with a SSL_read call from the perspective of the program as the client is returning zero during the upload (which makes sense) but the way my proxy is handling connections the only reason it could have read is if there was a SSL_get_error() that returned a SSL_ERROR_WANT_READ code. snipped most code We need to distinguish SSL_read/write and socket level. You should, and it looks like your code does, call SSL_read when you want (app-level) data and either _WANT_READ was previously returned and select says readable (normal case) or _WANT_WRITE was returned and select says writable (only renegotiation, which I doubt gmail does, but others might). Similarly for write. It appears to me your psc-ssl_read should be called once at connection start; it might get data which you process, and schedule another read when select says; or it returns -1 and SSL_get_error returns _WANT_READ and you schedule the next read when select says. In either case 'scheduled' reads should normally return data and schedule again, although it is possible there is socket level data that does not constitute a complete app-level record, so SSL_read=-1/get_error=_WANT_READ, and you re-schedule. Or you could schedule_read(psc::ssl_read) at startup, so the first ssl_read is attempted when select says but without _WANT_READ having cause that scheduling. For a HTTP-unaware proxy, you must try to SSL_read from each side (and forward to the other) 'always', but can't when you don't have buffer space available. I don't see any checking, so I guess buf-len_avail() returns 0 and you call SSL_read with max 0, and it looks to me that will return 0 but NOT meaning disconnect/EOF, and I'm not sure what SSL_get_error does in this state; it may well 'default' to _SYSCALL. If the client=browser is 'nearby' in network terms e.g. on same LAN and gmail is 'distant', then it is quite plausible receiving is faster than sending and you do get buffer (temporarily) full unless it's big enough. void ProxySSLConnection::handle_ssl_error(int ret, handler_function handler, const char * caller) { int error = SSL_get_error(_ssl, ret); switch (error) { case SSL_ERROR_WANT_READ: schedule_read(handler); break; case SSL_ERROR_WANT_WRITE: schedule_write(handler); break; case SSL_ERROR_ZERO_RETURN: _proxy_thread-shutdown(); break; default: unsigned long err = ERR_get_error(); while (err != 0) { char* err_string = ERR_error_string(err, NULL); cout err_string endl; err = ERR_get_error(); } close_connection(); _proxy_thread-shutdown(); break; } } I suggest for ERROR_SYSCALL and probably ERROR_SSL it would often help to know what was going on when you encountered the error. Your (added, below) caller gives the operation; do you have info conveniently in (or possibly mappable from) your ProxySSLConnection that says which side this is, and if you handle multiple connections concurrently (which from your structure I guess is a goal/feature) which connection? Alternatively can you get a debugger on this (state)? Then you can look around at whatever you need (in memory). If the following case is added to the switch then the upload is fixed, but the threads never exit: case SSL_ERROR_SYSCALL: //This case is where the uploads fail. schedule_write(handler); break; The caller variable is just for debug purposes. As per Dave Thompson's request I was using it to determine which call was causing the error, and was then calling perror(caller) since I was getting a SSL_ERROR_SYSCALL with a ret of 0, and the perror printed read: Success. Did you do perror() before couterr_string could clobber errno? Assuming so, that's either disconnect-without-shutdown (because shutdown would have given _ZERO_RETURN) but neither
RE: Connection Resetting
Sam Jantz wrote: It's multi threaded with non-blocking I/O. I'm not sure exactly what you mean by socket discovery, but I think you are asking how my program determines when something is ready? If that's the case then my program uses a select statement to watch the file descriptor to see if it's ready for read or write. It uses a call back system to perform the correct action based on which fd_set was ready. Okay, just make sure to only call 'select' when OpenSSL tells you to. Otherwise, you may be waiting for something that has already happened. void ProxySSLConnection::handle_ssl_error(int ret, handler_function handler, const char * caller) { int error = SSL_get_error(_ssl, ret); switch (error) { case SSL_ERROR_WANT_READ: schedule_read(handler); break; Your code has a subtle race condition because it assumes the two directions of an SSL connection have independent states. Consider the following case: 1) SSL_read on connection A returns SSL_ERROR_WANT_READ. 2) In another thread, SSL_read on connection B returns with some data. 3) Some data arrives on connection A. SSL_read on connection A now would return data immediately. 4) You call SSL_write on connection A to send the data you received in step 2. It reads from the socket the data that arrived in step 3. (SSL_read would not return data without having to read on the socket, the socket is not readable.) 5) You now act on the SSL_ERROR_WANT_READ you got in step 1, but it was invalidated by the actions in step 4. You call 'select' to wait for data that has already been received and never see the data received in step 3 and read in step 4. Before you call 'select' to wait for readability or writability, you must make sure that data movement in the other direction did not make the WANT_READ/WANT_WRITE indication invalid. This bug tends to rear its ugly head only on renegotiations though. So I don't think it's causing your actual problem. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Connection Resetting
Okay so the fix for the bug that I mentioned before introduced a much worse bug (That's what I get for not knowing exactly what is going on). This new bug causes the system to keep all threads alive for the length of the proxy so with enough sites visited the computer the proxy is running on becomes useless (100% processor load). I now understand that the reason they never exit is because the connection is never getting shutdown properly as far as the TLS standard is concerned, so the fix I put in was to basically ignore that, and keep going. But alas, that didn't work for long. My question is why is the session getting reset before I can upload a file? Is there some sort of a watchdog timer that I am neglecting to poke before the connection is reset? Any help would be appreciated. Thanks, Sam On Mon, Aug 30, 2010 at 12:49 PM, Sam Jantz sjan...@gmail.com wrote: Dave, Thank you for the clarification on HTTP keep-alives. I have just now fixed the bug. The source of the problem was an SSL_read call on the client half of the proxy. This was triggering an error SSL_ERROR_SYSCALL with a ret of zero. According to the documentation this is normally caused by an improperly shutdown SSL connection, however rescheduling the read for when the socket was ready (using a select statement) fixed this issue. I have tested it up to a 5MB file, and it works perfectly. I am a little confused on why I was getting the error in the first place still though. What would cause SSL_ERROR_SYSCALL to be flagged, and have an empty error queue if the socket was not closed improperly on the other side? On Sun, Aug 29, 2010 at 11:06 PM, Dave Thompson dthomp...@prinpay.comwrote: From: owner-openssl-us...@openssl.org On Behalf Of Sam Jantz Sent: Friday, 27 August, 2010 18:16 I have a question concerning Keep-Alives. I'm writing a SSL proxy (which is working great except for this issue) and every time I [POST about 470KB rather than about 18KB] the connection resets, and it gets caught in an infinite retransmit loop. snip This behavior is only implemented in Firefox. In the other browsers it seems to fail out with some error about unexpected reset. Is there some parameter that I can set when establishing the SSL connection that will allow me to wait for larger transfers without reseting? 1. This has nothing to do with keep-alives. HTTP 1.1 keep-alive is a passive feature; it doesn't do anything, instead if agreed the server REFRAINS FROM closing the connection as it would for 1.0. 2. It sounds like the browser is getting RST. (Or to be exact, getting an error from the OS that *it* got RST.) Firefox might respond to this differently than other browsers, by retrying; I don't have time to test. If so, the RST is caused by your proxy doing something abnormal, most likely dying. Check your code for bugs, and/or your logs -- your program does have logging and diagnostic code in it, like any well-designed program, right? 3. Or do you think the proxy is getting RST from gmail? I am 99.99% certain google wouldn't have a problem that would do that, although it isn't completely impossible. It's much more likely to be some network (mis)feature between you and gmail, like a firewall, NAT box, access controller, transparent (but not really) cache, etc. Try without your proxy, but with a client (i.e. browser) on the machine where the proxy is, to the same server with the same amounts of data (or at least reasonably close). If you can, try from different places in the Internet, like from home or a Starbucks versus the company office. 4. SSL itself has no timelimits; it will wait forever, or until the underlying TCP connection fails. (If a remote host just dies without closing properly, TCP may detect this in anywhere from a few minutes to many hours or days, depending.) An application *using* SSL might have a timelimit; if so you have to look to that program as to how, and whether, you can change it. And sometimes a firewall or NAT box or such has an idle timeout, where it will terminate your connection if it isn't used for an excessive period of time, and some netadmins have a crazy idea what is excessive; but I've never seen less than 15 minutes, which I expect is not the case in your example. The really awful ones do this silently, or by faking FIN; the ones that fake(?) RST at least give you a detectable error. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org -- Sam Jantz Software Engineer -- Sam Jantz Software Engineer
RE: Connection Resetting
From: owner-openssl-us...@openssl.org On Behalf Of Sam Jantz Sent: Monday, 30 August, 2010 13:50 I have just now fixed the bug. The source of the problem was an SSL_read call on the client half of the proxy. This was triggering This is ambiguous; do you mean the connection to the client (where the proxy is acting as server) or the connection to the server (proxy acting as client)? The latter makes more sense to me, see below. an error SSL_ERROR_SYSCALL with a ret of zero. According to the documentation this is normally caused by an improperly shutdown SSL connection, however rescheduling the read for when the socket was ready (using a select statement) fixed this issue. I have tested it up to a 5MB file, and it works perfectly. This isn't entirely clear; are you saying SSL_read returned 0 (which indicates TCP disconnect aka EOF) and *then* SSL_get_error returned _SYSCALL (and not _ZERO_RETURN)? That would mean the peer disconnected but without doing a shutdown alert first. It's arguable whether this is improper, but it's at least suboptimal or possibly worrisome. You didn't say, and I didn't think to ask: how do you know you are sending the whole request? If you are say SSL_read'ing only the first 32K or 100K or 1M or such of the request from the client, SSL_write'ing that to the server, and then SSL_read'ing the server, of course you can't get a valid response. In that case I would expect gmail to time-out and disconnect -- as a huge public service, it can't afford to keep potentially huge numbers of connections from wedged clients. If they did just-disconnect, it might be politer to do SSL-shutdown, but depending on the software structure that might be difficult. You might have been on the right track originally about keepalive. For 1.0 you can simply do connection-oriented: while (n=SSL_read(fromcli,buf)) 0 SSL_write(tosvr,n); if n==0 /* EOF=disconnect=request complete, now do response */ while (n=SSL_read(fromsvr,buf)) 0 SSL_write(tocli,n); close(tocli); /* indicate end */ close(tosvr); /* clean up */ elif n==-1 /* error, possibly incomplete request */ ?? But for 1.1 if keepalive is enabled (and all browsers I have used do, although technically it is optional) you *won't* get EOF following (and delimiting) the request, so in general you must either: 1. parse the request headers (at least if there is a body, which there is for POST) and do: while more_req() (n=SSL_read(cli))0 SSL_write(svr) if error or incomplete ?? /* similarly for response if body, which there is for most requests, with the additional complication of possible chunked transport */ loop for next request+response, until EOF on either side or 2. do 'full-duplex' which works for any HTTP sequence: while forever or until manually interrupted when data available from cli read and write to svr when data available from svr read and write to cli in between do something that doesn't hog CPU but if an error happens you don't know what the HTTP state is and can't even try to recover. Using select-readable *on both sides* gives you a good approximation to this, but in general SSL and thus openssl may need to both send and receive even on a connection that is logically write-only or read-only, so instead of just select'ing for readable (or writable), the robust way is to use nonblocking sockets, try SSL_read or SSL_write, let it return -1 and SSL_get_error will tell you _WANT_READ or _WANT_WRITE, and (remember and) select for that. This is described in both the SSL_read/write and SSL_get_error manpages. Or, less efficient but simpler, just (re)try _read (and _write when needed) every X milliseconds, and it will progress when it can. I am a little confused on why I was getting the error in the first place still though. What would cause SSL_ERROR_SYSCALL to be flagged, and have an empty error queue if the socket was not closed improperly on the other side? First, EOF isn't really an error. Second, when SSL_read etc. (calls BIO_sock which) gets a socket error, it returns -1 and SSL_get_error returns _SYSCALL, but the error is not (usually?) put in the ERR_ queue. You must instead use errno on Unix or [WSA]GetLastError() on Windows. The manpage for SSL_get_error says this may be the case and in my experience it always is. (Note that internally, at the OS level, [WSA]EWOULDBLOCK/etc. for nonblocking are treated as errors, but openssl handles them internally so your code only sees 'real' errors.) __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Connection Resetting
I'm writing a SSL proxy (which is working great except for this issue) and every time I got to attach a file in an email the connection resets, and it gets caught in an infinite retransmit loop. There are two totally different ways you can make an SSL proxy, and to figure out your issue, we really need to know which type. 1) An SSL proxy can understand the underlying protocol, know which side is supposed to transmit when, and only try to read from that side. In this case, it's vital that the proxy correctly track the protocol and not be reading from one side when it's the other side's turn to send. 2) An SSL proxy can ignore the underlying protocol and not know which side is supposed to transmit when. In this case, the proxy must always be ready to read from either side. It must never block indefinitely trying to read from one side. You can also have a hybrid. For example, you can read only from the client side until you get the full request, and then once you process the request, you switch to bidirectional proxying. It is very common for people to naively assume that their code will magically know which side to read from. I assure, this is not the case. Unless you carefully track the protocol, all you know is that the client has to send some data first. But once it does, all bets are off -- again, unless you carefully track the protocol. Also, you don't mention whether your I/O is blocking or non-blocking, and if non-blocking, how your socket discovery works. This can be subtle with OpenSSL and your mistake might lie there. For example, if you using blocking I/O, you can't just block one thread in SSL_read in each direction, because if you do, there's nothing you can do when SSL_read returns (since the connection you need to send on is in use, potentially indefinitely, by the other thread). DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Connection Resetting
Dave, Thank you for the clarification on HTTP keep-alives. I have just now fixed the bug. The source of the problem was an SSL_read call on the client half of the proxy. This was triggering an error SSL_ERROR_SYSCALL with a ret of zero. According to the documentation this is normally caused by an improperly shutdown SSL connection, however rescheduling the read for when the socket was ready (using a select statement) fixed this issue. I have tested it up to a 5MB file, and it works perfectly. I am a little confused on why I was getting the error in the first place still though. What would cause SSL_ERROR_SYSCALL to be flagged, and have an empty error queue if the socket was not closed improperly on the other side? On Sun, Aug 29, 2010 at 11:06 PM, Dave Thompson dthomp...@prinpay.comwrote: From: owner-openssl-us...@openssl.org On Behalf Of Sam Jantz Sent: Friday, 27 August, 2010 18:16 I have a question concerning Keep-Alives. I'm writing a SSL proxy (which is working great except for this issue) and every time I [POST about 470KB rather than about 18KB] the connection resets, and it gets caught in an infinite retransmit loop. snip This behavior is only implemented in Firefox. In the other browsers it seems to fail out with some error about unexpected reset. Is there some parameter that I can set when establishing the SSL connection that will allow me to wait for larger transfers without reseting? 1. This has nothing to do with keep-alives. HTTP 1.1 keep-alive is a passive feature; it doesn't do anything, instead if agreed the server REFRAINS FROM closing the connection as it would for 1.0. 2. It sounds like the browser is getting RST. (Or to be exact, getting an error from the OS that *it* got RST.) Firefox might respond to this differently than other browsers, by retrying; I don't have time to test. If so, the RST is caused by your proxy doing something abnormal, most likely dying. Check your code for bugs, and/or your logs -- your program does have logging and diagnostic code in it, like any well-designed program, right? 3. Or do you think the proxy is getting RST from gmail? I am 99.99% certain google wouldn't have a problem that would do that, although it isn't completely impossible. It's much more likely to be some network (mis)feature between you and gmail, like a firewall, NAT box, access controller, transparent (but not really) cache, etc. Try without your proxy, but with a client (i.e. browser) on the machine where the proxy is, to the same server with the same amounts of data (or at least reasonably close). If you can, try from different places in the Internet, like from home or a Starbucks versus the company office. 4. SSL itself has no timelimits; it will wait forever, or until the underlying TCP connection fails. (If a remote host just dies without closing properly, TCP may detect this in anywhere from a few minutes to many hours or days, depending.) An application *using* SSL might have a timelimit; if so you have to look to that program as to how, and whether, you can change it. And sometimes a firewall or NAT box or such has an idle timeout, where it will terminate your connection if it isn't used for an excessive period of time, and some netadmins have a crazy idea what is excessive; but I've never seen less than 15 minutes, which I expect is not the case in your example. The really awful ones do this silently, or by faking FIN; the ones that fake(?) RST at least give you a detectable error. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org -- Sam Jantz Software Engineer
RE: Connection Resetting
From: owner-openssl-us...@openssl.org On Behalf Of Sam Jantz Sent: Friday, 27 August, 2010 18:16 I have a question concerning Keep-Alives. I'm writing a SSL proxy (which is working great except for this issue) and every time I [POST about 470KB rather than about 18KB] the connection resets, and it gets caught in an infinite retransmit loop. snip This behavior is only implemented in Firefox. In the other browsers it seems to fail out with some error about unexpected reset. Is there some parameter that I can set when establishing the SSL connection that will allow me to wait for larger transfers without reseting? 1. This has nothing to do with keep-alives. HTTP 1.1 keep-alive is a passive feature; it doesn't do anything, instead if agreed the server REFRAINS FROM closing the connection as it would for 1.0. 2. It sounds like the browser is getting RST. (Or to be exact, getting an error from the OS that *it* got RST.) Firefox might respond to this differently than other browsers, by retrying; I don't have time to test. If so, the RST is caused by your proxy doing something abnormal, most likely dying. Check your code for bugs, and/or your logs -- your program does have logging and diagnostic code in it, like any well-designed program, right? 3. Or do you think the proxy is getting RST from gmail? I am 99.99% certain google wouldn't have a problem that would do that, although it isn't completely impossible. It's much more likely to be some network (mis)feature between you and gmail, like a firewall, NAT box, access controller, transparent (but not really) cache, etc. Try without your proxy, but with a client (i.e. browser) on the machine where the proxy is, to the same server with the same amounts of data (or at least reasonably close). If you can, try from different places in the Internet, like from home or a Starbucks versus the company office. 4. SSL itself has no timelimits; it will wait forever, or until the underlying TCP connection fails. (If a remote host just dies without closing properly, TCP may detect this in anywhere from a few minutes to many hours or days, depending.) An application *using* SSL might have a timelimit; if so you have to look to that program as to how, and whether, you can change it. And sometimes a firewall or NAT box or such has an idle timeout, where it will terminate your connection if it isn't used for an excessive period of time, and some netadmins have a crazy idea what is excessive; but I've never seen less than 15 minutes, which I expect is not the case in your example. The really awful ones do this silently, or by faking FIN; the ones that fake(?) RST at least give you a detectable error. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Connection Resetting
Hello all, I have a question concerning Keep-Alives. I'm writing a SSL proxy (which is working great except for this issue) and every time I got to attach a file in an email the connection resets, and it gets caught in an infinite retransmit loop. I've tested it with Gmail, and also with the outlook here at the office. I can attach a small file (17KB tested) but when I venture to a larger file (say 469KB for the image I'm testing) the connection will never complete, and will continue to reset the transmission, and loop. This behavior is only implemented in Firefox. In the other browsers it seems to fail out with some error about unexpected reset. Is there some parameter that I can set when establishing the SSL connection that will allow me to wait for larger transfers without reseting? Here is a sample of what I mean, this is the SSL_write() method output (caution: long): POST /mail/?url omitted for safety HTTP/1.1 Host: mail.google.com User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.19) Gecko/2010072022 Iceweasel/3.0.6 (Debian-3.0.6-3) Firefox on x64 debian, fails on windows too Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: https://mail.google.com/mail/?ui=2view=bspver=ohhl4rw8mbn4 Cookie: omitted for privacy Content-Type: multipart/form-data; boundary=---671339629238101832741976743 Content-Length: 481618 -671339629238101832741976743 Content-Disposition: form-data; name=to jim someem...@gmail.com, -671339629238101832741976743 Content-Disposition: form-data; name=cc -671339629238101832741976743 Content-Disposition: form-data; name=bcc -671339629238101832741976743 Content-Disposition: form-data; name=subject Test Attach Again -671339629238101832741976743 Content-Disposition: form-data; name=f_gddktpet0; filename=mario.jpg Content-Type: image/jpeg FFD8FFE0 B3BDAA^YC9o{B1F4C0ZU+0526]DA^A^?87871^B/ACBE2^D9E6FAB0A9BBD4^_E1B9wC1CE+W BF[F6nBD7BDY9BF3zEE7`:EDE8{BBU+061Cy CED4F9EDqD0ތED[^Vʎ#CFD7^Q B9eF2y...@ajb^tA9uC5 And then the process repeats nearly word for word. However when I attach a smaller image it works perfectly and instead of repeating a get a server HTTP/1.1 200 OK message. Any information would be appreciated. Thank you. -Sam -- Sam Jantz Software Engineer