The inconclusive thread http://marc.theaimsgroup.com/?l=turbine-user&m=100434715718774&w=2 discussed a problem where turbine was initialised multiple times.
I recall seeing this problem myself in the past, but it just went away so I conveniently ignored it. Unfortunately the problems we ignore often come back to haunt us! I am once again seeing this problem off and on (mostly on). I have three turbine 2.1 environments all on the same hosted Linux box (identical bar a few caching parameters). Depending on some yet to be determined factor the services in these environments are being initialised 1 .. 6 times. As luck would have it the environment where the services are being initialised the most is a Production environment - as a result, for this environment there are multiple scheduler services and hence scheduled tasks are executing more than once! The complete contents of turbine.log for the environment that is initialising services twice is included at the end of this message. Notice how the services are initialised once by the thread [main] and then a second time by the thread [Thread-1]. Other thread numbers are used when additional initialisations occurr. E.g.: 2002-05-15 13:16:40,947 [main] INFO system - Added Mapping for Service: ResourceService 2002-05-15 13:20:29,942 [Thread-1] INFO system - Added Mapping for Service: ResourceService The occurrence of multiple initialisations appears to be user related. The evidence that points to this is the fact that the first initialisation coincides with the time tomcat is started for the [main] thread and the others are at intervals that co-inside with users logging in [Thread-???]. As the multiple service invocations stack up the jvm runs out of memory and users start experiencing java.lang.OutOfMemoryError - not very good. If anyone can provide a clue as to what is going on I would sincerely appreciate it. Thanks, Scott === complete turbine.log follows === 2002-05-15 13:16:40,942 [main] INFO system - ServiceBroker: LoggingService enabled. 2002-05-15 13:16:40,947 [main] INFO system - Added Mapping for Service: ResourceService 2002-05-15 13:16:40,947 [main] INFO system - Added Mapping for Service: LoggingService 2002-05-15 13:16:40,947 [main] INFO system - Added Mapping for Service: FactoryService 2002-05-15 13:16:40,948 [main] INFO system - Added Mapping for Service: PoolService 2002-05-15 13:16:40,948 [main] INFO system - Added Mapping for Service: RunDataService 2002-05-15 13:16:40,948 [main] INFO system - Added Mapping for Service: ServletService 2002-05-15 13:16:40,948 [main] INFO system - Added Mapping for Service: AssemblerBrokerService 2002-05-15 13:16:40,949 [main] INFO system - Added Mapping for Service: MimeTypeService 2002-05-15 13:16:40,949 [main] INFO system - Added Mapping for Service: GlobalCacheService 2002-05-15 13:16:40,949 [main] INFO system - Added Mapping for Service: SchedulerService 2002-05-15 13:16:40,949 [main] INFO system - Added Mapping for Service: UniqueIdService 2002-05-15 13:16:40,950 [main] INFO system - Added Mapping for Service: UploadService 2002-05-15 13:16:40,950 [main] INFO system - Added Mapping for Service: SecurityService 2002-05-15 13:16:40,950 [main] INFO system - Added Mapping for Service: PoolBrokerService 2002-05-15 13:16:40,950 [main] INFO system - Added Mapping for Service: MapBrokerService 2002-05-15 13:16:40,950 [main] INFO system - Added Mapping for Service: PullService 2002-05-15 13:16:40,951 [main] INFO system - Added Mapping for Service: IntakeService 2002-05-15 13:16:40,951 [main] INFO system - Added Mapping for Service: TemplateService 2002-05-15 13:16:40,951 [main] INFO system - Added Mapping for Service: VelocityService 2002-05-15 13:16:40,952 [main] INFO system - Initializing all services using: org.apache.catalina.core.StandardWrapperFacade 2002-05-15 13:16:40,952 [main] INFO system - Start Initializing service (early): FactoryService 2002-05-15 13:16:40,962 [main] INFO system - Finish Initializing service (early): FactoryService 2002-05-15 13:16:40,963 [main] INFO system - Start Initializing service (early): PoolService 2002-05-15 13:16:40,968 [main] INFO system - Finish Initializing service (early): PoolService 2002-05-15 13:16:40,969 [main] INFO system - Start Initializing service (early): RunDataService 2002-05-15 13:16:40,983 [main] INFO system - Finish Initializing service (early): RunDataService 2002-05-15 13:16:40,984 [main] INFO system - Start Initializing service (early): ServletService 2002-05-15 13:16:40,993 [main] INFO system - Finish Initializing service (early): ServletService 2002-05-15 13:16:40,993 [main] INFO system - Start Initializing service (early): AssemblerBrokerService 2002-05-15 13:16:40,999 [main] INFO system - Finish Initializing service (early): AssemblerBrokerService 2002-05-15 13:16:40,999 [main] INFO system - Start Initializing service (early): MimeTypeService 2002-05-15 13:16:41,005 [main] INFO system - Finish Initializing service (early): MimeTypeService 2002-05-15 13:16:41,005 [main] INFO system - Start Initializing service (early): GlobalCacheService 2002-05-15 13:16:41,020 [main] INFO system - Finish Initializing service (early): GlobalCacheService 2002-05-15 13:16:41,020 [main] INFO system - Start Initializing service (early): SchedulerService 2002-05-15 13:16:41,032 [main] INFO system - TurbineNonPersistentSchedulerService init()....starting! 2002-05-15 13:16:41,044 [main] INFO system - TurbineNonPersistentSchedulerService init()....finished! 2002-05-15 13:16:41,044 [main] INFO system - Finish Initializing service (early): SchedulerService 2002-05-15 13:16:41,045 [main] INFO system - Start Initializing service (early): UniqueIdService 2002-05-15 13:16:41,054 [main] INFO system - Finish Initializing service (early): UniqueIdService 2002-05-15 13:16:41,054 [main] INFO system - Start Initializing service (early): UploadService 2002-05-15 13:16:41,063 [main] INFO system - Finish Initializing service (early): UploadService 2002-05-15 13:16:41,064 [main] INFO system - Start Initializing service (early): SecurityService 2002-05-15 13:16:41,111 [main] INFO system - Finish Initializing service (early): SecurityService 2002-05-15 13:16:41,112 [main] INFO system - Start Initializing service (early): PoolBrokerService 2002-05-15 13:16:41,120 [main] INFO system - Finish Initializing service (early): PoolBrokerService 2002-05-15 13:16:41,120 [main] INFO system - Start Initializing service (early): MapBrokerService 2002-05-15 13:16:41,127 [main] INFO system - Finish Initializing service (early): MapBrokerService 2002-05-15 13:16:41,127 [main] INFO system - Start Initializing service (early): PullService 2002-05-15 13:16:41,135 [main] INFO system - Finish Initializing service (early): PullService 2002-05-15 13:16:41,136 [main] INFO system - Start Initializing service (early): IntakeService 2002-05-15 13:16:41,469 [main] INFO system - Finish Initializing service (early): IntakeService 2002-05-15 13:16:41,469 [main] INFO system - Start Initializing service (early): TemplateService 2002-05-15 13:16:41,480 [main] INFO system - Finish Initializing service (early): TemplateService 2002-05-15 13:16:41,480 [main] INFO system - Start Initializing service (early): VelocityService 2002-05-15 13:16:41,493 [main] INFO system - Start Initializing service (late): ServletService 2002-05-15 13:16:41,494 [main] INFO system - Finish Initializing service (late): ServletService 2002-05-15 13:16:44,080 [main] INFO system - Start Initializing service (late): PullService 2002-05-15 13:16:44,091 [main] INFO system - Instantiated tool class org.apache.turbine.services.pull.util.UIManager to add to the context as '$ui' 2002-05-15 13:16:44,119 [main] INFO system - Instantiated tool class com.backstagetech.turbine.util.template.TemplateLinkEx to add to the context as '$link' 2002-05-15 13:16:44,127 [main] INFO system - Instantiated tool class com.backstagetech.turbine.util.template.TemplatePageAttributesEx to add to the context as '$page' 2002-05-15 13:16:44,131 [main] INFO system - Instantiated tool class org.apache.turbine.util.ContentURI to add to the context as '$content' 2002-05-15 13:16:44,135 [main] INFO system - Instantiated tool class org.apache.turbine.om.OMTool to add to the context as '$om' 2002-05-15 13:16:44,140 [main] INFO system - Instantiated tool class org.apache.turbine.services.intake.IntakeTool to add to the context as '$intake' 2002-05-15 13:16:44,165 [main] INFO system - Instantiated tool class com.backstagetech.cmes.util.CmesPullTool to add to the context as '$cmes' 2002-05-15 13:16:44,168 [main] INFO system - Instantiated tool class com.backstagetech.cmes.util.ContentAdminTool to add to the context as '$ca' 2002-05-15 13:16:44,171 [main] INFO system - Finish Initializing service (late): PullService 2002-05-15 13:16:44,174 [main] INFO system - Start Initializing service (late): TemplateService 2002-05-15 13:16:44,175 [main] INFO system - Finish Initializing service (late): TemplateService 2002-05-15 13:16:44,176 [main] INFO system - Finish Initializing service (early): VelocityService 2002-05-15 13:16:44,176 [main] INFO system - Finished initializing all services! 2002-05-15 13:20:29,938 [Thread-1] INFO system - ServiceBroker: LoggingService enabled. 2002-05-15 13:20:29,942 [Thread-1] INFO system - Added Mapping for Service: ResourceService 2002-05-15 13:20:29,942 [Thread-1] INFO system - Added Mapping for Service: LoggingService 2002-05-15 13:20:29,943 [Thread-1] INFO system - Added Mapping for Service: FactoryService 2002-05-15 13:20:29,943 [Thread-1] INFO system - Added Mapping for Service: PoolService 2002-05-15 13:20:29,943 [Thread-1] INFO system - Added Mapping for Service: RunDataService 2002-05-15 13:20:29,943 [Thread-1] INFO system - Added Mapping for Service: ServletService 2002-05-15 13:20:29,945 [Thread-1] INFO system - Added Mapping for Service: AssemblerBrokerService 2002-05-15 13:20:29,945 [Thread-1] INFO system - Added Mapping for Service: MimeTypeService 2002-05-15 13:20:29,946 [Thread-1] INFO system - Added Mapping for Service: GlobalCacheService 2002-05-15 13:20:29,946 [Thread-1] INFO system - Added Mapping for Service: SchedulerService 2002-05-15 13:20:29,946 [Thread-1] INFO system - Added Mapping for Service: UniqueIdService 2002-05-15 13:20:29,946 [Thread-1] INFO system - Added Mapping for Service: UploadService 2002-05-15 13:20:29,947 [Thread-1] INFO system - Added Mapping for Service: SecurityService 2002-05-15 13:20:29,947 [Thread-1] INFO system - Added Mapping for Service: PoolBrokerService 2002-05-15 13:20:29,947 [Thread-1] INFO system - Added Mapping for Service: MapBrokerService 2002-05-15 13:20:29,947 [Thread-1] INFO system - Added Mapping for Service: PullService 2002-05-15 13:20:29,947 [Thread-1] INFO system - Added Mapping for Service: IntakeService 2002-05-15 13:20:29,948 [Thread-1] INFO system - Added Mapping for Service: TemplateService 2002-05-15 13:20:29,948 [Thread-1] INFO system - Added Mapping for Service: VelocityService 2002-05-15 13:20:29,948 [Thread-1] INFO system - Initializing all services using: org.apache.catalina.core.StandardWrapperFacade 2002-05-15 13:20:29,948 [Thread-1] INFO system - Start Initializing service (early): FactoryService 2002-05-15 13:20:29,957 [Thread-1] INFO system - Finish Initializing service (early): FactoryService 2002-05-15 13:20:29,967 [Thread-1] INFO system - Start Initializing service (early): PoolService 2002-05-15 13:20:29,972 [Thread-1] INFO system - Finish Initializing service (early): PoolService 2002-05-15 13:20:29,976 [Thread-1] INFO system - Start Initializing service (early): RunDataService 2002-05-15 13:20:29,984 [Thread-1] INFO system - Finish Initializing service (early): RunDataService 2002-05-15 13:20:29,985 [Thread-1] INFO system - Start Initializing service (early): ServletService 2002-05-15 13:20:30,014 [Thread-1] INFO system - Finish Initializing service (early): ServletService 2002-05-15 13:20:30,014 [Thread-1] INFO system - Start Initializing service (early): AssemblerBrokerService 2002-05-15 13:20:30,020 [Thread-1] INFO system - Finish Initializing service (early): AssemblerBrokerService 2002-05-15 13:20:30,021 [Thread-1] INFO system - Start Initializing service (early): MimeTypeService 2002-05-15 13:20:30,049 [Thread-1] INFO system - Finish Initializing service (early): MimeTypeService 2002-05-15 13:20:30,050 [Thread-1] INFO system - Start Initializing service (early): GlobalCacheService 2002-05-15 13:20:30,060 [Thread-1] INFO system - Finish Initializing service (early): GlobalCacheService 2002-05-15 13:20:30,060 [Thread-1] INFO system - Start Initializing service (early): SchedulerService 2002-05-15 13:20:30,085 [Thread-1] INFO system - TurbineNonPersistentSchedulerService init()....starting! 2002-05-15 13:20:30,102 [Thread-1] INFO system - TurbineNonPersistentSchedulerService init()....finished! 2002-05-15 13:20:30,102 [Thread-1] INFO system - Finish Initializing service (early): SchedulerService 2002-05-15 13:20:30,102 [Thread-1] INFO system - Start Initializing service (early): UniqueIdService 2002-05-15 13:20:30,113 [Thread-1] INFO system - Finish Initializing service (early): UniqueIdService 2002-05-15 13:20:30,113 [Thread-1] INFO system - Start Initializing service (early): UploadService 2002-05-15 13:20:30,125 [Thread-1] INFO system - Finish Initializing service (early): UploadService 2002-05-15 13:20:30,125 [Thread-1] INFO system - Start Initializing service (early): SecurityService 2002-05-15 13:20:30,246 [Thread-1] INFO system - Finish Initializing service (early): SecurityService 2002-05-15 13:20:30,247 [Thread-1] INFO system - Start Initializing service (early): PoolBrokerService 2002-05-15 13:20:30,255 [Thread-1] INFO system - Finish Initializing service (early): PoolBrokerService 2002-05-15 13:20:30,255 [Thread-1] INFO system - Start Initializing service (early): MapBrokerService 2002-05-15 13:20:30,261 [Thread-1] INFO system - Finish Initializing service (early): MapBrokerService 2002-05-15 13:20:30,261 [Thread-1] INFO system - Start Initializing service (early): PullService 2002-05-15 13:20:30,269 [Thread-1] INFO system - Finish Initializing service (early): PullService 2002-05-15 13:20:30,269 [Thread-1] INFO system - Start Initializing service (early): IntakeService 2002-05-15 13:20:30,563 [Thread-1] INFO system - Finish Initializing service (early): IntakeService 2002-05-15 13:20:30,564 [Thread-1] INFO system - Start Initializing service (early): TemplateService 2002-05-15 13:20:30,571 [Thread-1] INFO system - Finish Initializing service (early): TemplateService 2002-05-15 13:20:30,571 [Thread-1] INFO system - Start Initializing service (early): VelocityService 2002-05-15 13:20:30,584 [Thread-1] INFO system - Start Initializing service (late): ServletService 2002-05-15 13:20:30,584 [Thread-1] INFO system - Finish Initializing service (late): ServletService 2002-05-15 13:20:32,783 [Thread-1] INFO system - Start Initializing service (late): PullService 2002-05-15 13:20:32,790 [Thread-1] INFO system - Instantiated tool class org.apache.turbine.services.pull.util.UIManager to add to the context as '$ui' 2002-05-15 13:20:32,816 [Thread-1] INFO system - Instantiated tool class com.backstagetech.turbine.util.template.TemplateLinkEx to add to the context as '$link' 2002-05-15 13:20:32,824 [Thread-1] INFO system - Instantiated tool class com.backstagetech.turbine.util.template.TemplatePageAttributesEx to add to the context as '$page' 2002-05-15 13:20:32,828 [Thread-1] INFO system - Instantiated tool class org.apache.turbine.util.ContentURI to add to the context as '$content' 2002-05-15 13:20:32,832 [Thread-1] INFO system - Instantiated tool class org.apache.turbine.om.OMTool to add to the context as '$om' 2002-05-15 13:20:32,837 [Thread-1] INFO system - Instantiated tool class org.apache.turbine.services.intake.IntakeTool to add to the context as '$intake' 2002-05-15 13:20:32,861 [Thread-1] INFO system - Instantiated tool class com.backstagetech.cmes.util.CmesPullTool to add to the context as '$cmes' 2002-05-15 13:20:32,864 [Thread-1] INFO system - Instantiated tool class com.backstagetech.cmes.util.ContentAdminTool to add to the context as '$ca' 2002-05-15 13:20:32,867 [Thread-1] INFO system - Finish Initializing service (late): PullService 2002-05-15 13:20:32,870 [Thread-1] INFO system - Start Initializing service (late): TemplateService 2002-05-15 13:20:32,871 [Thread-1] INFO system - Finish Initializing service (late): TemplateService 2002-05-15 13:20:32,871 [Thread-1] INFO system - Finish Initializing service (early): VelocityService 2002-05-15 13:20:32,872 [Thread-1] INFO system - Finished initializing all services! 2002-05-15 13:20:33,378 [Thread-1] INFO system - Start Initializing service (late): RunDataService 2002-05-15 13:20:33,379 [Thread-1] INFO system - Finish Initializing service (late): RunDataService 2002-05-15 13:20:33,379 [Thread-1] INFO system - Start Initializing service (late): PoolService 2002-05-15 13:20:33,380 [Thread-1] INFO system - Finish Initializing service (late): PoolService 2002-05-15 13:20:33,382 [Thread-1] INFO system - Start Initializing service (late): FactoryService 2002-05-15 13:20:33,383 [Thread-1] INFO system - Finish Initializing service (late): FactoryService 2002-05-15 13:20:33,417 [Thread-1] INFO system - Initializing all services using: org.apache.turbine.services.rundata.DefaultTurbineRunData 2002-05-15 13:20:33,417 [Thread-1] INFO system - Start Initializing service (early): FactoryService 2002-05-15 13:20:33,417 [Thread-1] INFO system - Finish Initializing service (early): FactoryService 2002-05-15 13:20:33,417 [Thread-1] INFO system - Start Initializing service (early): PoolService 2002-05-15 13:20:33,417 [Thread-1] INFO system - Finish Initializing service (early): PoolService 2002-05-15 13:20:33,418 [Thread-1] INFO system - Start Initializing service (early): RunDataService 2002-05-15 13:20:33,418 [Thread-1] INFO system - Finish Initializing service (early): RunDataService 2002-05-15 13:20:33,418 [Thread-1] INFO system - Start Initializing service (early): ServletService 2002-05-15 13:20:33,418 [Thread-1] INFO system - Finish Initializing service (early): ServletService 2002-05-15 13:20:33,418 [Thread-1] INFO system - Start Initializing service (early): AssemblerBrokerService 2002-05-15 13:20:33,419 [Thread-1] INFO system - Finish Initializing service (early): AssemblerBrokerService 2002-05-15 13:20:33,419 [Thread-1] INFO system - Start Initializing service (early): MimeTypeService 2002-05-15 13:20:33,419 [Thread-1] INFO system - Finish Initializing service (early): MimeTypeService 2002-05-15 13:20:33,419 [Thread-1] INFO system - Start Initializing service (early): GlobalCacheService 2002-05-15 13:20:33,419 [Thread-1] INFO system - Finish Initializing service (early): GlobalCacheService 2002-05-15 13:20:33,419 [Thread-1] INFO system - Start Initializing service (early): SchedulerService 2002-05-15 13:20:33,420 [Thread-1] INFO system - Finish Initializing service (early): SchedulerService 2002-05-15 13:20:33,420 [Thread-1] INFO system - Start Initializing service (early): UniqueIdService 2002-05-15 13:20:33,427 [Thread-1] INFO system - This is Turbine instance running at: http://staging.changedtoprotecttheinnocent.com.au/elearning/go 2002-05-15 13:20:33,427 [Thread-1] INFO system - The instance id is #E942EC20A3023B0AAD2304F73CA8F8CE 2002-05-15 13:20:33,427 [Thread-1] INFO system - Finish Initializing service (early): UniqueIdService 2002-05-15 13:20:33,428 [Thread-1] INFO system - Start Initializing service (early): UploadService 2002-05-15 13:20:33,428 [Thread-1] INFO system - Finish Initializing service (early): UploadService 2002-05-15 13:20:33,428 [Thread-1] INFO system - Start Initializing service (early): SecurityService 2002-05-15 13:20:33,428 [Thread-1] INFO system - Finish Initializing service (early): SecurityService 2002-05-15 13:20:33,428 [Thread-1] INFO system - Start Initializing service (early): PoolBrokerService 2002-05-15 13:20:33,429 [Thread-1] INFO system - Finish Initializing service (early): PoolBrokerService 2002-05-15 13:20:33,429 [Thread-1] INFO system - Start Initializing service (early): MapBrokerService 2002-05-15 13:20:33,429 [Thread-1] INFO system - Finish Initializing service (early): MapBrokerService 2002-05-15 13:20:33,429 [Thread-1] INFO system - Start Initializing service (early): PullService 2002-05-15 13:20:33,429 [Thread-1] INFO system - Finish Initializing service (early): PullService 2002-05-15 13:20:33,429 [Thread-1] INFO system - Start Initializing service (early): IntakeService 2002-05-15 13:20:33,430 [Thread-1] INFO system - Finish Initializing service (early): IntakeService 2002-05-15 13:20:33,430 [Thread-1] INFO system - Start Initializing service (early): TemplateService 2002-05-15 13:20:33,430 [Thread-1] INFO system - Finish Initializing service (early): TemplateService 2002-05-15 13:20:33,430 [Thread-1] INFO system - Start Initializing service (early): VelocityService 2002-05-15 13:20:33,430 [Thread-1] INFO system - Finish Initializing service (early): VelocityService 2002-05-15 13:20:33,430 [Thread-1] INFO system - Finished initializing all services! 2002-05-15 13:20:33,436 [Thread-1] INFO system - Start Initializing service (late): AssemblerBrokerService 2002-05-15 13:20:33,455 [Thread-1] INFO system - Finish Initializing service (late): AssemblerBrokerService 2002-05-15 13:20:33,464 [Thread-1] INFO system - Start Initializing service (late): UploadService 2002-05-15 13:20:33,464 [Thread-1] INFO system - Finish Initializing service (late): UploadService -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
