Heath-- That sounds like a great plan. I must say I've learned way more about NTLM and TCP dumps than I ever wanted, although it's been interesting. Sure would be helpful if MS clearly documented their protocols instead of making people reverse-engineer and make educated guesses... Anyway, I look forward to giving it a shot next week sometime (Monday at the earliest when I'm in the office). Thanks for all your help!
[I've written some software myself and I know how sometimes even the smallest little thing has to be bought with a huge price of time and effort, sometimes on the part of several people, so I really appreciate what you, Bruce, and others are doing here to help track this down.] --Steve On 9/17/04 2:51 AM, "Heath Raftery" <[EMAIL PROTECTED]> wrote: > Steven, others, > > I've been browsing through the dump for a while now, and have tried to > equate it with the code. It is very revealing, but not absolutely clear > yet. > > As you say, the 407 response comes back immediately, as it should. Your > computer receives (acknowledges) the request, but something is fishy. > Each of the dumps has a duplicate ACK (acknowledge) on the last packet > of the response. This should not be so! For those following along at > home, this is Frame 57 in networktraffic, Frame 96 in networktraffic1 > and Frame 25 in networktraffic2. > > From what I've read, a duplicate acknowledge indicates a lost packet. > But the packet is not lost - the dump clearly shows the packets > arriving intact. I can't figure out why the duplication acknowledge is > sent. Anyway, at this point the server is supposed to drop the > connection, and that's what Authoxy waits for (it doesn't predict the > end of the connection, it waits for the server to close it). The server > does not close the connection at this point, and hence the delay. > > Now there are three possibilities that have come to mind in my > experimentation, as to the reason the connection is not closed: > > 1. The duplicate ACK is confusing the server. It is not sure whether > the client has received all the data, or what it expects, so it keeps > the connection open. Since I don't know why the dup is sent in the > first place, I'm at a bit of a loss to solve that one. > > 2. I noticed the 407 the proxy returns is quite large, and splits over > a number of frames. The frames happen to be bigger than the receive > buffer Authoxy uses internally (1448 bytes of data in frame compared to > 1024 internal buffer), and I suddenly wondered whether this backlog of > data is the cause of the duplicate ACK. To test the case, I just wrote > a small server and client which emulates the Authoxy<->proxy > connection, and despite sending 2000 bytes in one go, the client > buffered the data without a hitch, and the packet dump looked clean. > > 3. Finally, I just reread the NTLM protocol specification, and noticed > that one source says that after sending the 407 response, the server > "Typically, the server closes the connection at this time". Note the > word "Typically". I deliberately wrote that section of code to wait for > the server's closure of the connection, because it simplifies and > speeds the process by not having to interpret the data that is coming > in. In order to predict the end of the data transfer, Authoxy would > have to find the content length header and count the bytes as they came > in. Since all my testing indicated that the server closed the > connection after sending the data, I didn't bother trying to predict > the closure. > > The second traffic dump shows that if the connection is manually broken > (hitting the cancel button) the authentication process appears to > continue okay. Of course, the client is no longer expecting it to > continue, so this is not going to work in the long run. > > In conclusion, there seem to be a few things we can try. Over the > weekend I will work at making three changes to that section of the NTLM > authentication code, and give you a binary to try. I'll change the > following: > > 1. Internal buffer size to 2048 bytes. An ethernet frame is typically > 1514 bytes, leaving about 1448 bytes left for data, so I think an > internal buffer of 2k is a nice trade-off. Not too much wasted, and > less chance of having the kernel buffer the data. > 2. I noticed some of the buffer copying code is that section is > slightly inefficient. I'll make a small change to tidy that up. > 3. I'll change the code that receives the 407 to predict the close of > connection. That is, it will find the Content-Length header, and count > the bytes that come in. When enough bytes have been received, Authoxy > will close the connection, regardless of whether the server has > already, and continue with the rest of the authentication. The code to > enable this already exists in other parts of the application, so > *hopefully* it should go smoothly! > > Give me a day or two, and I'll see what I come up with. > Heath > > On 15/09/2004, at 11:33 PM, Steven Stratford wrote: > >> networktraffic is a complete traffic dump for a Software Update. My >> computer >> is 10.2.2.44, the Proxy server is 10.2.0.2 >> >> networktraffic1 is a dump of Software Update that is interrupted by >> turning >> Authoxy off after about 10 seconds. >> >> networktraffic2 is a dump of Software Update that is interrupted by >> canceling Software Update after about 10 seconds. >> >> I looked over the dumps with Ethereal. Looks to me like a "HTTP/1.1 407 >> Proxy Authentication Required" response is received from 10.2.0.2 >> immediately, but then I could be reading it incorrectly. >> >> --Steve
