Environment: Windows Server 2008
IIS 7.0 Tomcat 6.0.29 ISAPI Redirect JK Connector 1.2.30 At 14:06:57 this afternoon, IIS performed a recycle (it does this ever 29 hours and has for years without causing us problems): “A worker process with process id of '10536' serving application pool 'DefaultAppPool' has requested a recycle because the worker process reached its allowed processing time limit.” The last log entry in the IIS log file until the recycle complete was at 14:06:57. We get about 100 requests per second, and there were about 100 requests at 14:06:57. >From 14:06:58-14:07:01 (the three seconds following the recycle), the following log messages appeared in the JK connector ISAPI redirect log (more information following this abbreviated log output): [Fri May 20 14:06:23.707 2011] [10536:7896] [error] ajp_service::jk_ajp_common.c (2559): (s01aspgrp03) connecting to tomcat failed. [Fri May 20 14:06:58.745 2011] [12064:6756] [error] wc_create_worker::jk_worker.c (151): factory for lb failed for lbaspgrp10 [Fri May 20 14:06:58.854 2011] [12064:6756] [error] build_worker_map::jk_worker.c (262): failed to create worker lbaspgrp10 [Fri May 20 14:06:58.854 2011] [12064:6756] [error] uri_worker_map_ext::jk_uri_worker_map.c (506): Could not find worker with name 'lbaspgrp14' in uri map post processing. … (hundreds of these messages) … [Fri May 20 14:06:59.386 2011] [12064:6756] [error] uri_worker_map_ext::jk_uri_worker_map.c (506): Could not find worker with name 'lbaspgrp13' in uri map post processing. [Fri May 20 14:06:59.433 2011] [12064:8708] [error] ajp_worker_factory::jk_ajp_common.c (2929): allocating ajp worker record from shared memory [Fri May 20 14:06:59.433 2011] [12064:8708] [error] wc_create_worker::jk_worker.c (151): factory for ajp13 failed for default [Fri May 20 14:06:59.433 2011] [12064:8708] [error] build_worker_map::jk_worker.c (262): failed to create worker default [Fri May 20 14:06:59.433 2011] [12064:8708] [error] uri_worker_map_ext::jk_uri_worker_map.c (506): Could not find worker with name 'lbaspgrp14' in uri map post processing. … (hundreds of these messages) … [Fri May 20 14:07:00.012 2011] [12064:8708] [error] uri_worker_map_ext::jk_uri_worker_map.c (506): Could not find worker with name 'lbaspgrp13' in uri map post processing. [Fri May 20 14:07:00.043 2011] [12064:14296] [error] ajp_worker_factory::jk_ajp_common.c (2929): allocating ajp worker record from shared memory [Fri May 20 14:07:00.043 2011] [12064:14296] [error] wc_create_worker::jk_worker.c (151): factory for ajp13 failed for default [Fri May 20 14:07:00.043 2011] [12064:14296] [error] build_worker_map::jk_worker.c (262): failed to create worker default [Fri May 20 14:07:00.043 2011] [12064:14296] [error] uri_worker_map_ext::jk_uri_worker_map.c (506): Could not find worker with name 'lbaspgrp14' in uri map post processing. … (hundreds of these messages) … [Fri May 20 14:07:00.606 2011] [12064:14296] [error] uri_worker_map_ext::jk_uri_worker_map.c (506): Could not find worker with name 'lbaspgrp13' in uri map post processing. [Fri May 20 14:07:00.638 2011] [12064:14904] [error] ajp_worker_factory::jk_ajp_common.c (2929): allocating ajp worker record from shared memory [Fri May 20 14:07:00.638 2011] [12064:14904] [error] wc_create_worker::jk_worker.c (151): factory for ajp13 failed for default [Fri May 20 14:07:00.653 2011] [12064:14904] [error] build_worker_map::jk_worker.c (262): failed to create worker default [Fri May 20 14:07:00.653 2011] [12064:14904] [error] uri_worker_map_ext::jk_uri_worker_map.c (506): Could not find worker with name 'lbaspgrp14' in uri map post processing. … (hundreds of these messages) … [Fri May 20 14:07:01.169 2011] [12064:14904] [error] uri_worker_map_ext::jk_uri_worker_map.c (506): Could not find worker with name 'lbaspgrp13' in uri map post processing. Beginning at 14:06:59 and continuing until we shut down IIS and failed over to another server 30 minutes later, requests began appearing in the IIS log file again, but at this time ALL requests were replied to with a 500 internal server error. We have been running this server with this configuration for over a year without any issues like this until now. No configuration settings were changed. No workers were added. We do not have shm_size set anywhere since, per the documentation, that directive is not needed anymore as of 1.2.27. I have been unable to find any useful information about this elsewhere, save this bug: https://issues.apache.org/bugzilla/show_bug.cgi?id=40877 However, that bug was apparently fixed back in 2007 with version 1.2.20. So, I’m not sure if it’s related or not. Any insights? Has anyone else seen this? Is this an IIS problem or a new/reoccurring JK Connector bug or a configuration problem? Thanks in advance! Nick
