Re: w3wp randomly crashes when redirector is used
Hmmm. Are you sure that you are posting to the right list here ? Steve Ryan wrote: Hello I really hope someone can help me with this. I have been trying to figure it out for days and I am beginning to go mad! The issue I am having appears to be very similar to https://issues.apache.org/bugzilla/show_bug.cgi?id=45063. It might even be the same issue. Note I can recreate the problem on various flavours of Windows, and in IIS 6 IIS 7. Details of the issue: I have a web application which lives under the Default Web Site in IIS 6, and which has its own app pool. The redirector (isapi_redirect.dll 1.2.28) lives under the DefaultAppPool. The user would log into our web application (which uses our app pool) and everything works as expected. If he then uses functionality which uses the redirector (and the DefaultAppPool), w3wp.exe will probably crash. (I say probably because sometimes it works without error). Note everything works fine (i.e. there is no crash) if I run the DefaultAppPool as Local System; the problem only seems to exist if I run the DefaultAppPool as Local Service or Network Service. Details of the crash: In the Event Viewer I can see the following - Faulting application w3wp.exe, version 6.0.3790.3959, faulting module ntdll.dll, version 5.2.3790.4455, fault address 0x0004f053. And if I catch the crash using Debug Diagnostic Tool - Thread 5 - System ID 6140 Entry point w3tp!THREAD_MANAGER::ThreadManagerThread Create time 01/02/2010 13:13:26 Time spent in user mode 0 Days 0:0:0.0 Time spent in kernel mode 0 Days 0:0:0.15 Function Arg 1 Arg 2 Arg 3 Source ntdll!RtlpWaitOnCriticalSection+c5 4b66d376 ntdll!RtlEnterCriticalSection+152 6a6ee4e8 6a6b9ae6 isapi_redirect!TerminateFilter+c24b 76bbcc54 isapi_redirect+9ae6 01459180 4b66d376 isapi_redirect!TerminateFilter+21435 1000 01b6dd18 isapi_redirect!HttpExtensionProc+29c 732448ee 73243b7a iisutil!STRU::STRU+c 73243b7a 0157e69c 7327a93f nativerd!CONFIG_ELEMENT::ReferenceConfigElement+f 000cf7f0 7327a970 nativerd!CONFIG_ELEMENT_LIST::GetMergedElement+483 73243e08 0157eaa8 0x0157e68c NTDLL!RTLPWAITONCRITICALSECTION+C5In w3wp__PID__5276__Date__02_01_2010__Time_01_13_27PM__226__Second_Chance_Excep tion_C005.dmp the assembly instruction at ntdll!RtlpWaitOnCriticalSection+c5 in C:\Windows\System32\ntdll.dll from Microsoft Corporation has caused an access violation exception (0xC005) when trying to write to memory location 0x0014 on thread 5 Module Information Image Name: C:\Windows\System32\ntdll.dll Base address: 0x76fc Checksum: 0x00135d86 COM DLL: False ISAPIExtension: False ISAPIFilter: False Managed DLL: False VB DLL: False Loaded Image Name: ntdll.dll Mapped Image Name: Module name: ntdll Single Threaded: False Module Size: 1.15 MBytes Symbol File Name: c:\symcache\ntdll.pdb\B958B2F91A5A46B889DAFAB4D140CF252\ntdll.pdb Symbol Type: PDB Time Stamp: Sat Jan 19 07:32:54 2008 Comments: Company Name: Microsoft Corporation File Description: NT Layer DLL File Version: 6.0.6001.18000 (longhorn_rtm.080118-1840) Internal Name: ntdll.dll Legal Copyright: C Microsoft Corporation. All rights reserved. Legal Trademarks: Original filename: ntdll.dll Private Build: Product Name: MicrosoftR WindowsR Operating System Product Version: 6.0.6001.18000 Special Build: There is nothing obvious in Process Monitor which explains what is causing the error. Do any of you have any ideas what could be causing this? Any advice would be greatly appreciated. Cheers Steve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: w3wp randomly crashes when redirector is used
Thanks for the reply. My issue is due to Tomcat Connectors 1.2.28. Is there a more specific mailing list for such issues? The Tomcat Connectors website shows users@tomcat.apache.org as the mailing list. Cheers Steve -Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: 03 February 2010 20:44 To: Tomcat Users List Subject: Re: w3wp randomly crashes when redirector is used Hmmm. Are you sure that you are posting to the right list here ? Steve Ryan wrote: Hello I really hope someone can help me with this. I have been trying to figure it out for days and I am beginning to go mad! The issue I am having appears to be very similar to https://issues.apache.org/bugzilla/show_bug.cgi?id=45063. It might even be the same issue. Note I can recreate the problem on various flavours of Windows, and in IIS 6 IIS 7. Details of the issue: I have a web application which lives under the Default Web Site in IIS 6, and which has its own app pool. The redirector (isapi_redirect.dll 1.2.28) lives under the DefaultAppPool. The user would log into our web application (which uses our app pool) and everything works as expected. If he then uses functionality which uses the redirector (and the DefaultAppPool), w3wp.exe will probably crash. (I say probably because sometimes it works without error). Note everything works fine (i.e. there is no crash) if I run the DefaultAppPool as Local System; the problem only seems to exist if I run the DefaultAppPool as Local Service or Network Service. Details of the crash: In the Event Viewer I can see the following - Faulting application w3wp.exe, version 6.0.3790.3959, faulting module ntdll.dll, version 5.2.3790.4455, fault address 0x0004f053. And if I catch the crash using Debug Diagnostic Tool - Thread 5 - System ID 6140 Entry point w3tp!THREAD_MANAGER::ThreadManagerThread Create time 01/02/2010 13:13:26 Time spent in user mode 0 Days 0:0:0.0 Time spent in kernel mode 0 Days 0:0:0.15 Function Arg 1 Arg 2 Arg 3 Source ntdll!RtlpWaitOnCriticalSection+c5 4b66d376 ntdll!RtlEnterCriticalSection+152 6a6ee4e8 6a6b9ae6 isapi_redirect!TerminateFilter+c24b 76bbcc54 isapi_redirect+9ae6 01459180 4b66d376 isapi_redirect!TerminateFilter+21435 1000 01b6dd18 isapi_redirect!HttpExtensionProc+29c 732448ee 73243b7a iisutil!STRU::STRU+c 73243b7a 0157e69c 7327a93f nativerd!CONFIG_ELEMENT::ReferenceConfigElement+f 000cf7f0 7327a970 nativerd!CONFIG_ELEMENT_LIST::GetMergedElement+483 73243e08 0157eaa8 0x0157e68c NTDLL!RTLPWAITONCRITICALSECTION+C5In w3wp__PID__5276__Date__02_01_2010__Time_01_13_27PM__226__Second_Chance_Excep tion_C005.dmp the assembly instruction at ntdll!RtlpWaitOnCriticalSection+c5 in C:\Windows\System32\ntdll.dll from Microsoft Corporation has caused an access violation exception (0xC005) when trying to write to memory location 0x0014 on thread 5 Module Information Image Name: C:\Windows\System32\ntdll.dll Base address: 0x76fc Checksum: 0x00135d86 COM DLL: False ISAPIExtension: False ISAPIFilter: False Managed DLL: False VB DLL: False Loaded Image Name: ntdll.dll Mapped Image Name: Module name: ntdll Single Threaded: False Module Size: 1.15 MBytes Symbol File Name: c:\symcache\ntdll.pdb\B958B2F91A5A46B889DAFAB4D140CF252\ntdll.pdb Symbol Type: PDB Time Stamp: Sat Jan 19 07:32:54 2008 Comments: Company Name: Microsoft Corporation File Description: NT Layer DLL File Version: 6.0.6001.18000 (longhorn_rtm.080118-1840) Internal Name: ntdll.dll Legal Copyright: C Microsoft Corporation. All rights reserved. Legal Trademarks: Original filename: ntdll.dll Private Build: Product Name: MicrosoftR WindowsR Operating System Product Version: 6.0.6001.18000 Special Build: There is nothing obvious in Process Monitor which explains what is causing the error. Do any of you have any ideas what could be causing this? Any advice would be greatly appreciated. Cheers Steve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.733 / Virus Database: 271.1.1/2665 - Release Date: 02/03/10 08:09:00 - To unsubscribe, e-mail: users
RE: w3wp randomly crashes when redirector is used
From: Steve Ryan [mailto:st...@acme.ie] Subject: RE: w3wp randomly crashes when redirector is used Is there a more specific mailing list for such issues? No, this is the proper mailing list. Unfortunately, the problem is sufficiently esoteric (as well as having a work-around) so you're not likely to get much response from the user community. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: w3wp randomly crashes when redirector is used
Hi Chuck Thanks for your reply. It is indeed a strange problem. The work around unfortunately causes a severe security risk (giving the DefaultAppPool the Local System identity is similar to running Apache as root) so it isn't really an option. If someone who works on the Tomcat Connectors project is reading this, do you think this might be the same issue as bug 45063? If so, do you have any sort of idea when 45063 might be looked at? Thanks again for taking the time to read respond to my e-mails. Cheers Steve -Original Message- From: Caldarale, Charles R [mailto:chuck.caldar...@unisys.com] Sent: 03 February 2010 21:47 To: Tomcat Users List Subject: RE: w3wp randomly crashes when redirector is used From: Steve Ryan [mailto:st...@acme.ie] Subject: RE: w3wp randomly crashes when redirector is used Is there a more specific mailing list for such issues? No, this is the proper mailing list. Unfortunately, the problem is sufficiently esoteric (as well as having a work-around) so you're not likely to get much response from the user community. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.733 / Virus Database: 271.1.1/2665 - Release Date: 02/03/10 08:09:00 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: w3wp randomly crashes when redirector is used
Steve Ryan wrote: My issue is due to Tomcat Connectors 1.2.28. Is there a more specific mailing list for such issues? The Tomcat Connectors website shows users@tomcat.apache.org as the mailing list. No. If your issue is indeed due to the Windows version of the mod_jk connector, then you are on the correct list. I am far from the specialist on the Windows/jk side of things, but I am still puzzled however. According to your explanation below, the application which crashes is something called w3wp.exe. This does not really sound like an application which lives under Java and Tomcat. The jk redirector's function is to redirect (or rather proxy) some HTTP requests from IIS to a back-end Tomcat, so that these requests are processed by a Tomcat-based application. Conversely, the response from this Tomcat-based application is then returned through the jk redirector, back to IIS, and from there to the client. What motivates my puzzlement is that, if w3wp.exe is really an application which lives under IIS and not Tomcat, then there does not seem to exist any reason why the jk redirector would have anything to do with it. My knowledge being limited, there might well be something I am missing in all of this. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: w3wp randomly crashes when redirector is used
From: André Warnier [mailto:a...@ice-sa.com] Subject: Re: w3wp randomly crashes when redirector is used My knowledge being limited, there might well be something I am missing in all of this. All I'm going on is Mark T's comment on the bug report: Moving to TC6 to increase visibility. Apparently, it still isn't that visible. Perhaps the OP should add another example to the bug entry. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: w3wp randomly crashes when redirector is used
Hi Andre Thanks for your reply. W3wp.exe is the process which manages the application pool where the redirector lives. I'm fairly confident the problem is due to the redirector because the crash only occurs when the redirector is used. Also, there is an open bug (45063) which appears to be a very similar issue. I do of course accept the problem could lie at Microsoft's door though. :) I guess I'm hoping someone has heard of this issue before, and knows of some sort of workaround which doesn't involve using the Local System account. Cheers Steve -Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: 03 February 2010 21:58 To: Tomcat Users List Subject: Re: w3wp randomly crashes when redirector is used Steve Ryan wrote: My issue is due to Tomcat Connectors 1.2.28. Is there a more specific mailing list for such issues? The Tomcat Connectors website shows users@tomcat.apache.org as the mailing list. No. If your issue is indeed due to the Windows version of the mod_jk connector, then you are on the correct list. I am far from the specialist on the Windows/jk side of things, but I am still puzzled however. According to your explanation below, the application which crashes is something called w3wp.exe. This does not really sound like an application which lives under Java and Tomcat. The jk redirector's function is to redirect (or rather proxy) some HTTP requests from IIS to a back-end Tomcat, so that these requests are processed by a Tomcat-based application. Conversely, the response from this Tomcat-based application is then returned through the jk redirector, back to IIS, and from there to the client. What motivates my puzzlement is that, if w3wp.exe is really an application which lives under IIS and not Tomcat, then there does not seem to exist any reason why the jk redirector would have anything to do with it. My knowledge being limited, there might well be something I am missing in all of this. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.733 / Virus Database: 271.1.1/2665 - Release Date: 02/03/10 08:09:00 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: w3wp randomly crashes when redirector is used
Steve, I'm with you - the issue does look unpleasantly similar. I wonder whether some experimentation with running the redirector under a local account with particular rights set might tell you which right was required, and hence which one might be causing an issue? But in the longer term, the stack trace shows it's faulting in the redirector's TerminateFilter, while waiting on a critical section. Looking at this plus the identical stack trace in Bugzilla, it does feel like it's an issue with the redirector. Does it always happen in your app, or is it intermittent? If intermittent, is it more likely to trigger when something's under load? - Peter On 3 February 2010 22:12, Steve Ryan st...@acme.ie wrote: Hi Andre Thanks for your reply. W3wp.exe is the process which manages the application pool where the redirector lives. I'm fairly confident the problem is due to the redirector because the crash only occurs when the redirector is used. Also, there is an open bug (45063) which appears to be a very similar issue. I do of course accept the problem could lie at Microsoft's door though. :) I guess I'm hoping someone has heard of this issue before, and knows of some sort of workaround which doesn't involve using the Local System account. Cheers Steve -Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: 03 February 2010 21:58 To: Tomcat Users List Subject: Re: w3wp randomly crashes when redirector is used Steve Ryan wrote: My issue is due to Tomcat Connectors 1.2.28. Is there a more specific mailing list for such issues? The Tomcat Connectors website shows users@tomcat.apache.org as the mailing list. No. If your issue is indeed due to the Windows version of the mod_jk connector, then you are on the correct list. I am far from the specialist on the Windows/jk side of things, but I am still puzzled however. According to your explanation below, the application which crashes is something called w3wp.exe. This does not really sound like an application which lives under Java and Tomcat. The jk redirector's function is to redirect (or rather proxy) some HTTP requests from IIS to a back-end Tomcat, so that these requests are processed by a Tomcat-based application. Conversely, the response from this Tomcat-based application is then returned through the jk redirector, back to IIS, and from there to the client. What motivates my puzzlement is that, if w3wp.exe is really an application which lives under IIS and not Tomcat, then there does not seem to exist any reason why the jk redirector would have anything to do with it. My knowledge being limited, there might well be something I am missing in all of this. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.733 / Virus Database: 271.1.1/2665 - Release Date: 02/03/10 08:09:00 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
