Dave Phelan wrote: > What I see is that the server load remains pretty consistent, > it's the number of requests completed that is dropping.
Precisely, but in marked stages Dave Phelan wrote: > This could be an indication that the pages become more complex > as the process continues, or it may indicate that the faster > running threads are all completed leaving the longer running > threads at the end. Its neither I'm afraid. I have two jobs I am doing this in. One builds image pages for for-sale vehicles in a multiple listing service. Each listing can, at random, have one, two or dozens of images and thus dozens of pages needing to be built in its thread. Lots of variability there but I am not measuring thread completion rates. I'm watching the speed of the page requests, which should remain roughly constant over time. Each page is pretty much identical to the next in terms of CF load. All single-record key lookups. More telling: I have a separate job that creates the vehicle listing pages - the parent of the aforementioned image pages. This is always a 1-page thread/request, is always roughly the same job/load, and only 14,000-20,000 pages are made vs. 350,000-400,000. This smaller routine evidences the same staged decline in performance. The completion rates are different as the nature of the pages are different, but FusionReactor shows the same staged decline in output. Dave Phelan wrote: > My suggestion would be to do some time monitoring on the loops. > I think you'll find that the average completion time for each > thread increases as the number of completed requests decreases. I did that, and more on that in a bit, but I must be missing your point on rates. Thread completion time is going to be widely variable as one thread is spawned per vehicle listing, and each can have any number of images (from one to a couple hundred, averaging I believe around 40). What I am measuring however is the rate of page completion and that should - I would think - remain constant across the job unless some resource somewhere is hitting a limit. Loop monitoring: The original test code stored a db record at the start and stop of each loop completed. The 'stop' record gave me a count of how many pages were made and let me calc duration. That much db activity slowed things down so the job was slower than its single-threaded predecessor. In the end all it showed me was that things were slowing down proportionately, in stages, just like FusionReactor pointed out in its request list. > Is this apache or IIS? IIS 7.5 On Fri, Jul 26, 2013 at 12:05 PM, Mark A Kruger <[email protected]>wrote: > > Is this apache or IIS? > > -----Original Message----- > From: Money Pit [mailto:[email protected]] > Sent: Friday, July 26, 2013 11:41 AM > To: cf-talk > Subject: cfthread execution slowdown > > > I need another set of eyeballs on this. I can't see what I'm doing wrong. > > I'm building many thousands of static pages. I turned to cfthread to speed > things up. Each thread can output one page or more than one depending on > info retrieved. I use an array to manage the number of threads in use at > any one time. > > It works great, but I am seeing CF slow down - in fixed stages - throughout > the life of the process. Take a look at this image taken from > FusionReactor to see what I mean: > > http://205.210.189.205/images/screenshot081.jpg > > Its not a curve but rather three distinct stages of output. Stage 1 is > about 2900 pages per minute, Stage 2 is around 2400 and Stage 3 - where the > routine stabilizes - is about 1200. > > On a second server building differently-composed pages, I see exactly the > same thing. > > CF isn't bogging down from load... its flat out not working as hard. Its > as if it decided it didn't want to work as hard, so it eases off. Never > seen anything like it. > > Here's the code in a nutshell. Can anyone see something I am doing that is > causing this work slowdown? > > <cfset variables.threadArray=arrayNew(1)> > <cfset variables.threadCount=4> > <cfquery > name="getData" > datasource="#variables.dsn#"> > [sql goes here] > </cfquery> > <cfset variables.loopCounter=0> > <cfloop > condition="variables.loopCounter LT getData.recordCount"> > <cfset variables.threadsLive=arrayLen(variables.threadArray)> > <cfif variables.threadsLive lt variables.threadCount> > <cfset variables.loopCounter=variables.loopCounter+1> > <cfset variables.thisThreadID=createUUID()> > <cfset > temp=arrayAppend(variables.threadArray,variables.thisThreadID)> > <cfthread > name="#variables.thisThreadID#" > action="run"> > <!--- > cfhttp pulls local CF pages and stores as html, > queries determine if child pages need building etc. > ---> > <cfset > > temp=arrayDeleteAt(variables.threadArray,arrayFindNoCase(variables.threadArr > ay,variables.thisThreadID))> > </cfthread> > <!--- > may or may not do a cfthread join if I want to retrieve > variables from inside of the thread > ---> > </cfif> > </cfloop> > > > -- > --m@Robertson-- > Janitor, The Robertson Team > mysecretbase.com > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:356343 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
