Re: debugging ssl packet drop
Hi, I resent the mail since I didn't receive a copy of my first mail and I am subscribed to this list as well. In any case, I am not asking for help to debug any network issues here (if you read the content of my mail). There's an issue with a SSL packet being sent from IE10 browsers in the context of the websocket protocol (over ssl) and I have been working on the steps you mention further down in your email. My original mail was to see if I can get help regarding specific breakpoints in the code flow of the apache server/ssl module flow to check where the packet was being dropped. However, I'll take your suggestion that this probably needs to go to the openssl mailing list. Thank you. Regards, Umapathy On Fri, Oct 17, 2014 at 9:31 PM, Joe Lewis jle...@silverhawk.net wrote: Your first message was delivered less than 24 hours ago - most of us are not paid by the Apache modules developers list, meaning we are stricly volunteer, and 24 hours might not be enough time. I would suggest patience, especially while asking questions on the fringes of this lists expertise. Most people here are module developers, not SSL debuggers or TCP experts. I actually thought your original e-mail should have gone to an openssl mailing list instead of an Apache modules list. I won't support tracking down something like network errors without access and a consultation fee - it's an ugly rabbit hole that not many actually want to go down, especially me. I will simply suggest using Wireshark at multiple points (e.g. on the same LAN as the client, and on the same LAN as the server) just to ensure that a firewall, netscaler, or any other device between the client and the server isn't your problem. You claim an error code of 101 (network reset). Are you seeing TCP resets in your packet capture (remember, I'm not going to support or help beyond this - the question is to get you to think about what you are actually seeing in your packet capture). Did you decrypt your SSL-encrypted packet capture just to ensure you are seeing things properly? Are you sure you haven't custom-configured timeouts for Apache? ipfw/iptables/etc on the server (tcpdump of *nix will help)? Remember, it sounds like you are asking for help for things on the fringes of most people's expertise here. Be patient. Joe On Fri, Oct 17, 2014 at 9:22 AM, Pon Umapathy Kailash S pon.umapa...@gmail.com wrote: Resending since it doesn't seem to have been delivered. On Thu, Oct 16, 2014 at 11:26 AM, Pon Umapathy Kailash S pon.umapa...@gmail.com wrote: Hi, I am facing the an issue where a SSL packet from IE10 doesn't reach the client processing thread for a particular connection(more details below). Can you please provide me pointers on where to look/add more debug logs in the code to figure out what's happening? We use mpm worker threads. I have added support for websockets in a customised manner(as required for our application) inside apache. At a high level, it's done as follows: - the initial GET request with 101 code is handled by a handler hook function which computes the required security keys and sends back the response. Also, the socket on which the request came in is not closed(by maintaining a list and patching some parts of the apache code to not close if a socket is present in this list). - the child thread which processes this connection will relinquish the connection after the keep-alive timeout , which is ok since all we need is for the server to send messages to the client, with one exception. - At this point, the socket is recognised as a websocket client which is not yet authenticated(since from browsers we cannot set custom headers with the initial websocket connection request). - Authentication is done by the client sending the cookie as the 1st(and only) message on this connection to the server within the keep-alive timeout period(at which point the cookie is authenticated and the socket is marked as a valid, authenticated subscriber). (* there are other functions/timers to take care of stale, unauth connections etc) This works fine in all browsers with support for websockets with the following exceptions: IE10 over ssl(https/wss) and IE11 over ssl on 32-bit client machines. Doing a packet capture, we could figure out that the initial connection goes through fine and when the cookie is sent from the client, it reaches the server(and there's a tcp ack received at the client for this packet). However, the client processing this connection doesn't seem to receive this packet(this is well within the keep alive interval and the client thread is still actively processing that connection). Can you please let me know at which points in the code flow it might be useful to add debugging info to see where this is getting dropped? Regards, Umapathy
Re: debugging ssl packet drop
Another possible list thst would be good would be the HTTPD development list. You may need the author of mod_ssl. Thanks, Joe Lewis Original message From: Pon Umapathy Kailash S pon.umapa...@gmail.com Date: 10/18/2014 4:28 AM (GMT-07:00) To: modules-dev@httpd.apache.org Subject: Re: debugging ssl packet drop Hi, I resent the mail since I didn't receive a copy of my first mail and I am subscribed to this list as well. In any case, I am not asking for help to debug any network issues here (if you read the content of my mail). There's an issue with a SSL packet being sent from IE10 browsers in the context of the websocket protocol (over ssl) and I have been working on the steps you mention further down in your email. My original mail was to see if I can get help regarding specific breakpoints in the code flow of the apache server/ssl module flow to check where the packet was being dropped. However, I'll take your suggestion that this probably needs to go to the openssl mailing list. Thank you. Regards, Umapathy On Fri, Oct 17, 2014 at 9:31 PM, Joe Lewis jle...@silverhawk.net wrote: Your first message was delivered less than 24 hours ago - most of us are not paid by the Apache modules developers list, meaning we are stricly volunteer, and 24 hours might not be enough time. I would suggest patience, especially while asking questions on the fringes of this lists expertise. Most people here are module developers, not SSL debuggers or TCP experts. I actually thought your original e-mail should have gone to an openssl mailing list instead of an Apache modules list. I won't support tracking down something like network errors without access and a consultation fee - it's an ugly rabbit hole that not many actually want to go down, especially me. I will simply suggest using Wireshark at multiple points (e.g. on the same LAN as the client, and on the same LAN as the server) just to ensure that a firewall, netscaler, or any other device between the client and the server isn't your problem. You claim an error code of 101 (network reset). Are you seeing TCP resets in your packet capture (remember, I'm not going to support or help beyond this - the question is to get you to think about what you are actually seeing in your packet capture). Did you decrypt your SSL-encrypted packet capture just to ensure you are seeing things properly? Are you sure you haven't custom-configured timeouts for Apache? ipfw/iptables/etc on the server (tcpdump of *nix will help)? Remember, it sounds like you are asking for help for things on the fringes of most people's expertise here. Be patient. Joe On Fri, Oct 17, 2014 at 9:22 AM, Pon Umapathy Kailash S pon.umapa...@gmail.com wrote: Resending since it doesn't seem to have been delivered. On Thu, Oct 16, 2014 at 11:26 AM, Pon Umapathy Kailash S pon.umapa...@gmail.com wrote: Hi, I am facing the an issue where a SSL packet from IE10 doesn't reach the client processing thread for a particular connection(more details below). Can you please provide me pointers on where to look/add more debug logs in the code to figure out what's happening? We use mpm worker threads. I have added support for websockets in a customised manner(as required for our application) inside apache. At a high level, it's done as follows: - the initial GET request with 101 code is handled by a handler hook function which computes the required security keys and sends back the response. Also, the socket on which the request came in is not closed(by maintaining a list and patching some parts of the apache code to not close if a socket is present in this list). - the child thread which processes this connection will relinquish the connection after the keep-alive timeout , which is ok since all we need is for the server to send messages to the client, with one exception. - At this point, the socket is recognised as a websocket client which is not yet authenticated(since from browsers we cannot set custom headers with the initial websocket connection request). - Authentication is done by the client sending the cookie as the 1st(and only) message on this connection to the server within the keep-alive timeout period(at which point the cookie is authenticated and the socket is marked as a valid, authenticated subscriber). (* there are other functions/timers to take care of stale, unauth connections etc) This works fine in all browsers with support for websockets with the following exceptions: IE10 over ssl(https/wss) and IE11 over ssl on 32-bit client machines. Doing a packet capture, we could figure out that the initial connection goes through fine and when the cookie is sent from the client, it reaches the server(and there's a tcp ack received at the client for this packet). However, the client processing this connection doesn't seem to receive this packet(this is well
