So... You are creating a total of 1300 threads with this code?
I would be surprised if you could ever get that to run successfully. Some
things to try. You can check the length of a running thread and terminate it
if it runs past a certain time. You could add a sleep to the loop to slow
down the spawning of threads.
This (untested) example makes the loop sleep for 2 seconds after the
spawning of every 10 threads. This would add 260 seconds to your request but
would likely create enough margin for threads to complete so the thread
queue doesn't build too high.
</cfthread>
<cfif NOT currentRow MOD 10>
<cset sleep(2000)/>
</cfif>
</cfloop>
Mark A. Kruger, CFG, MCSE
(402) 408-3733 ext 105
www.cfwebtools.com
www.coldfusionmuse.com
www.necfug.com
-----Original Message-----
From: Ian Skinner [mailto:[EMAIL PROTECTED]
Sent: Friday, June 13, 2008 11:31 AM
To: CF-Talk
Subject: JVM Tuning and <cthread>s... I'm Back
HISTORY:
We have this new application to generate ~1300 pdf letters using <cfreport>
functionality. To streamline the processing, <cfthread> tags where used to
asynchronously process 10 reports at a time, witting each finished report to
a file. This feature worked just fine during small scale testing creating
up to 100 reports. But when first attempting to run all 1300 reports the
ColdFusion came crashing down with JRun out-of-memory errors.
We dived into the CF8 server monitor to see if we could get a hint on what
was actually happening. The first thing I see is that in 'REQUEST
STATISTICS > Active ColdFusion Threads" is that all the threads created by
this process never died. Repeat a run and more threads are created, but
none of them die. Then looking at "MEMORY USAGE > Memory Usage Summary" we
could see that as these perputual threads where created more and more memory
was being gobbled up by the JVM and never being released.
So this started us down a path to tune our JVM to make better use of the
servers resources and throttle the process so that these 1300 reports could
be made with these resources. We added some code that seemed to cause the
threads to be closed, but the JVM memory would still grow and grow until
failure. We then made some tweeks to jvm.config suggested in the previous
discussion and the first thing is that another process, that previously
worked fine, stopped running. I restored the original default JVM settings
and this other process started working again. I then, piece by piece,
modified the JVM configuration as suggested and got something that does not
prevent the running of current code. But not our problem code is creating
un-dieing threads again.
CURRENT STATE:
jvm.config
----------
# Arguments to VM
java.args=-server -Xmx1024m -Xms1024m -Dsun.io.useCanonCaches=false
-XX:MaxPermSize=192m -XX:PermSize=192m -server -XX:+UseParallelGC
-Dsun.rmi.dgc.client.gcInterval=60000 -server
-Dsun.rmi.dgc.server.gcInterval=60000
-Dcoldfusion.rootDir={application.home}/
allren.cfm
---------
<cfinclude template = "scripts.cfm">
<cfset Application.DSN = LIC_DSN>
<cfset pathDate = #DatePart("yyyy", Now()).toString()#/> <cfset pIssueDate =
#PadDateSegment(DatePart("m", Now()))# & "/" &
#PadDateSegment(DatePart("d", Now()))# & "/" &
#DatePart("yyyy", Now()).toString().substring(2)#>
<!--- GET RENEWAL ID --->
<cfstoredproc procedure="REPORT_UTILS.getBatchRenewals"
dataSource = "#LIC_DSN#">
<cfprocresult name = ResultFirms>
</cfstoredproc>
<!--- REPORT --->
<cfset threadList = "">
<cfloop index = "i" from="1" to="#ResultFirms.recordcount#">
<cfset threadList = listAppend(threadList,"batchRenew_thread#i#")>
<cfthread name="batchRenew_thread#i#" threadIndex="#i#" action="run">
<!--- GET RENEWAL ID --->
<cfset vRenewalSeq = ""/>
<cfstoredproc procedure="REPORT_UTILS.getRenewalSequenceId"
dataSource = "#LIC_DSN#">
<cfprocparam type="out" variable="vRenewalSeq"
CFSQLType="CF_SQL_VARCHAR">
</cfstoredproc>
<!--- PATH --->
<cfset pathDate = #DatePart("yyyy", Now()).toString()#/>
<cfset vOutputFilePath = RENEWALS_FILE_PATH/>
<cfset vOutputFilePath &= pathDate & "\"/>
<cfset vOutputFilePath &=
Trim(#ResultFirms["firmno"][threadIndex]#) & "\"/>
<!--- CREATE OUTPUT PATH --->
<cfif DirectoryExists(#vOutputFilePath#) is False>
<cfdirectory action="create" directory=#vOutputFilePath#/>
</cfif>
<!--- FILE NAME --->
<cfset vOutputFilePath &= GetFileName()/>
<!--- BUILD REPORT --->
<cfreport template="ren.cfr" format="pdf" overwrite="yes"
filename=#vOutputFilePath#>
<cfreportparam name="pFirmNo"
value=#ResultFirms["firmno"][threadIndex]#>
<cfreportparam name="pIssueDate" value=#DateFormat(pIssueDate,
'mm/dd/yy')#>
<cfreportparam name="pRenewalId" value=#vRenewalSeq#>
</cfreport>
<!--- INSERT RENEWAL --->
<cfstoredproc procedure="REPORT_UTILS.insertRenewal"
dataSource = "#LIC_DSN#">
<cfprocparam type="in" value=#vRenewalSeq#
CFSQLType="CF_SQL_NUMERIC">
<cfprocparam type="in"
value=#ResultFirms["firmno"][threadIndex]# CFSQLType="CF_SQL_NUMERIC">
<cfprocparam type="in" value=#pathDate# CFSQLType="CF_SQL_NUMERIC">
</cfstoredproc>
<cfstoredproc procedure="REPORT_UTILS.insertRenewalProductsForFirm"
dataSource = "#LIC_DSN#">
<cfprocparam type="in" value=#vRenewalSeq#
CFSQLType="CF_SQL_NUMERIC">
<cfprocparam type="in"
value=#ResultFirms["firmno"][threadIndex]# CFSQLType="CF_SQL_NUMERIC">
</cfstoredproc>
</cfthread>
</cfloop>
<cfoutput>#replace(listSort(structKeyList(cfthread),'text'),',','<br/>','ALL
')#</cfoutput>
<cfthread action="join" name="#structKeyList(cfthread)#"/>
<cfloop list="#structKeyList(cfthread)#" index="thread">
<cfthread action="terminate" name="#thread#"/> </cfloop>
<p>BLAH <cfoutput>#timeformat(now(),'HH:MM:SS:l')#</cfoutput></p>
Anybody have any suggestions?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to
date
Get the Free Trial
http://ad.doubleclick.net/clk;192386516;25150098;k
Archive:
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:307423
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
