RE: CFThread Memory Issues sending emails
For a quick fix, I'd split those up into smaller batches. For instance, your cfthread could process 1000 emails (or whatever number you decide is manageable by your configuration) id make the count a parameter that is easily passed in via URL so all you'd have to do is change the url var in a scheduled task to increase/decrease the count. Then the thread's process would be something like... 1) Grab top X records that need to be sent. 2) Send the emails for those top X records 3) Update those records to indicate that they have been processed so they arent grabbed again by step 1. 4) Stop. Then just schedule it to run every minute or so. If you use timestamps as your step 3 indicator, you still have the start and stop times once there are no emails left to send. For a more elaborate solution, id queue individual emails up into a DB table and have multiple threads process them one at a time until they were done (makeshift CF message queue). I've found that letting a thread process 100s of thousands of small items individually makes that thread much happier than if it had to process even a fraction of that amount all at once. That said, I would have to agree and think your described setup should have zero problem with sending that many emails in a single loop. Have you monitored CF to see how the thread was doing? Maybe watch jconsole memory stats as the thread is running to see if it is indeed climbing rapidly and causing the GC? Maybe the thread is being spawned more than once? Maybe you could try loading all of the used variables into a single struct within that thread so you can clear that struct at the end of the loop? I'm just throwing out ideas here... .:.:.:.:.:.:.:.:.:.:.:.:.:. Bobby Hartsfield http://acoderslife.com http://cf4em.com -Original Message- From: Firewall [mailto:firew...@cc.uk.com] Sent: Thursday, May 19, 2011 7:38 AM To: cf-talk Subject: CFThread Memory Issues sending emails This one has me really stumped. I'm running a pretty simple cfmail script to send out email messages to a user list. Up until now I was simply doing it with a browser reload but the client recently doubled the number of emails they want to send to about 57,000. So I decided to implement this with cfthread wrapped around my loop containing the cfquery and cfmail code. The idea being that the server simply returns the page to the browser and kicks of a massive send job in the background. Once the mail script completes an email is send to the user saying I sent your email to X people, I started at xx:xx and finished at xx:xx My testing worked perfectly and the client used it on smaller email jobs without a problem, I tried it for the first time yesterday on the main massive list and about a third of the messages were sent out before the server seemed to hang. Looking at the logs I saw loads of JVM errors referring to GC aka the garbage collector. The specific log entry was: java.lang.OutOfMemoryError: GC overhead limit exceeded The server seems to have no problems sending about 10-15k emails using this process but when I try it on this one big list it results in all kinds of problems so I'm wondering if this is an issue with cfthread, JVM, CF 8.0.1 or my code. I'm wondering if this anyone else has tried to do this and seen the same problem; and if so how did you go about solving it? My setup is Coldfusion 8.0.1 (64bit) running on Red Hat Enterprise 5, the server itself is a nearly new core i7 Dell PowerEdge with Raid SSD hard drives and 12Gb RAM; in other words it should look at 57k emails and just laugh at the smallness of the list :) I've included the outline of my send code just in case someone spots something really stupid in my code: cfthread name=sendEmail#Form.id# action=run cfset startingTime = '#timeformat(now(), HH:MM:SS)#' cfquery datasource=#Application.ds# name=gn Grab info to send in email /cfquery cfloop list=#Form.sendToList# index=e cfoutput cfquery datasource=#Application.ds# name=gu Grab user details based on ID contained in e /cfquery cftry cfmail to=#emailSendTo# from=#gn.addName# #gn.addEmail# subject=#gn.msgSubj# type=html cfinclude template=email-body-#gn.emailType#.cfm /cfmail cfcatch type=any div style=color:red; Failed send: #gu.email# - #cfcatch.message#br / /div /cfcatch /cftry /cfoutput /cfloop cfset endingTime = '#timeformat(now(), HH:MM:SS)#' cfmail
RE: cgi.host_name Security Exploit
Actually, I've seen that number work on sites even in 'production' mode as well. Indeed. .:.:.:.:.:.:.:.:.:.:.:.:.:. Bobby Hartsfield http://acoderslife.com http://cf4em.com -Original Message- From: Raymond Camden [mailto:rcam...@gmail.com] Sent: Thursday, May 19, 2011 12:10 PM To: cf-talk Subject: Re: cgi.host_name Security Exploit What about an ecommerce system that hits the test ecom system when in dev mode? If I knew your code did that, or suspected, I'd try it and use one of the many common test CC numbers, like 4111. Actually, I've seen that number work on sites even in 'production' mode as well. On Sat, May 14, 2011 at 4:51 AM, Russ Michaels r...@michaels.me.uk wrote: although I can't really think how spoofing the host_name would do any harm. In any of my apps all it does it determine whether to use live or dev settings which would only cause an error if the host name was wrong. On Fri, May 13, 2011 at 10:10 PM, Jason Durham jqdur...@gmail.com wrote: Dave pretty much summed it up. Anybody who knows what a HOSTS file is, knows how to mask the server_name. :) Jason Durham On Fri, May 13, 2011 at 2:28 PM, Dave Watts dwa...@figleaf.com wrote: I have recently discovered a security flaw that I have reported to the Adobe team regarding the use of the variable cgi.host_name. As you know, the cgi.host_name is typically the hostname of the server or the websites domain name. I've discovered an exploit that allows a user to basically change this variable to anything they want for the user's current session. This exploit could be spread across sessions in instances where a website is caching absolute links using the cgi.host_name variable. It could also be used to take advantage of applications that assume the cgi.host_name variable is a constant, therefore developed applications don't take precautions to sanitize this variable before inserting it into a database could have issues. While it's a good thing you're telling people about this, I'm not sure I'd categorize it as a security flaw with CF, or even a security flaw in general. CF doesn't have anything to do with creating or validating many of the CGI variables. They're provided by the browser's HTTP request headers, or by the web server. CF just uses what it's given. Of course, those values are inherently untrustworthy and should always be sanitized. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344804 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: scoping
ALWAYS SCOPE! Quite I've been struggling with a site for a side job and no scoping whatsoever. I'd love to discuss this with the original developer. Although the site is such a kludge I suspect that now this person is a CF hater now. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344805 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: scoping
I know several CF people who love the language and been making a living off it for around a decade. None of them scope things nor do they even understand what scoping is. On Sat, May 21, 2011 at 4:52 PM, Larry Lyons larrycly...@gmail.com wrote: ALWAYS SCOPE! Quite I've been struggling with a site for a side job and no scoping whatsoever. I'd love to discuss this with the original developer. Although the site is such a kludge I suspect that now this person is a CF hater now. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344806 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Baffing date problem
I love CF but dates are definitely my Achilles heel.
I've dumped qEvents.TheDate and its formatted like {ts '2011-06-07
00:00:00.0'}
I've dumped dayview and its formatted like {ts '2011-06-07 00:00:00'}
I would have thought that as time stamps CF would see these the same despite
the 0 difference but the query returns no rows and I'm stumped. A cfdump of
qEvents shows the query is returning data correctly. Funny thing is this
works on a CF6 box running SQL2000 but fails on a CF8 box running SQL2005.
What I'm trying to do is do a check for qEvents that on a given day there
are or are not events for that day (thisday). Any advice greatly appreciated
- thank you.
cfset dayview = #createdate(year, month, thisday)#
cfquery name=ForToday dbtype=query
select *
from qEvents
where qEvents.TheDate = #dayview#
/cfquery
++
Kevin Parker
Advanced Imaging
e: webmas...@advancedimaging.com.au
w: www.advancedimaging.com.au
m: 0418 815 527
++
http://au.linkedin.com/in/krparker
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344807
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Baffing date problem
Kevin, Try this. select * from qEvents where DateDiff(d, qEvents.TheDate, #dayview#) = 0 -- Russ Michaels www.bluethunderinternet.com : Business hosting services solutions www.cfmldeveloper.com: Free ColdFusion developer hosting www.michaels.me.uk : my blog www.cfsearch.com : ColdFusion search engine ** *skype me* : russmichaels ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344808 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Baffing date problem
Quotes around #dayview# ?
cfquery name=ForToday dbtype=query
select *
from qEvents
where qEvents.TheDate = '#dayview#'
-Original Message-
From: Kevin Parker [mailto:tras...@internode.on.net]
Sent: 22 May 2011 03:16
To: cf-talk
Subject: Baffing date problem
I love CF but dates are definitely my Achilles heel.
I've dumped qEvents.TheDate and its formatted like {ts '2011-06-07
00:00:00.0'}
I've dumped dayview and its formatted like {ts '2011-06-07 00:00:00'}
I would have thought that as time stamps CF would see these the
same despite
the 0 difference but the query returns no rows and I'm stumped. A
cfdump of
qEvents shows the query is returning data correctly. Funny thing is this
works on a CF6 box running SQL2000 but fails on a CF8 box running SQL2005.
What I'm trying to do is do a check for qEvents that on a given day there
are or are not events for that day (thisday). Any advice greatly
appreciated
- thank you.
cfset dayview = #createdate(year, month, thisday)#
cfquery name=ForToday dbtype=query
select *
from qEvents
where qEvents.TheDate = #dayview#
/cfquery
++
Kevin Parker
Advanced Imaging
e: webmas...@advancedimaging.com.au
w: www.advancedimaging.com.au
m: 0418 815 527
++
http://au.linkedin.com/in/krparker
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344809
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Baffing date problem
Thank you - that looks elegant - but that took a dive too -
Encountered DateDiff ( d ,. Incorrect conditional expression, Expected one
of [like|null|between|in|comparison] condition
The query looks like SQL
select * from qEvents where DateDiff(d, qEvents.TheDate, {ts '2011-06-01
00:00:00'}) = 0
you can see it here if it helps
http://www.trsbok.com/_ncsi/index.cfm?month=6year=2011
++
Kevin Parker
Advanced Imaging
e: webmas...@advancedimaging.com.au
w: www.advancedimaging.com.au
m: 0418 815 527
++
http://au.linkedin.com/in/krparker
-Original Message-
From: Russ Michaels [mailto:r...@michaels.me.uk]
Sent: Sunday, 22 May 2011 12:24
To: cf-talk
Subject: Re: Baffing date problem
Kevin,
Try this.
select *
from qEvents
where DateDiff(d, qEvents.TheDate, #dayview#) = 0
--
Russ Michaels
www.bluethunderinternet.com : Business hosting services solutions
www.cfmldeveloper.com: Free ColdFusion developer hosting
www.michaels.me.uk : my blog
www.cfsearch.com : ColdFusion search engine
**
*skype me* : russmichaels
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344810
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Baffing date problem
I tried that - but I'll have another crack - thank you!!!
++
Kevin Parker
Advanced Imaging
e: webmas...@advancedimaging.com.au
w: www.advancedimaging.com.au
m: 0418 815 527
++
http://au.linkedin.com/in/krparker
-Original Message-
From: Jenny Gavin-Wear [mailto:jenn...@fasttrackonline.co.uk]
Sent: Sunday, 22 May 2011 12:33
To: cf-talk
Subject: RE: Baffing date problem
Quotes around #dayview# ?
cfquery name=ForToday dbtype=query
select *
from qEvents
where qEvents.TheDate = '#dayview#'
-Original Message-
From: Kevin Parker [mailto:tras...@internode.on.net]
Sent: 22 May 2011 03:16
To: cf-talk
Subject: Baffing date problem
I love CF but dates are definitely my Achilles heel.
I've dumped qEvents.TheDate and its formatted like {ts '2011-06-07
00:00:00.0'}
I've dumped dayview and its formatted like {ts '2011-06-07 00:00:00'}
I would have thought that as time stamps CF would see these the
same despite
the 0 difference but the query returns no rows and I'm stumped. A
cfdump of
qEvents shows the query is returning data correctly. Funny thing is this
works on a CF6 box running SQL2000 but fails on a CF8 box running SQL2005.
What I'm trying to do is do a check for qEvents that on a given day there
are or are not events for that day (thisday). Any advice greatly
appreciated
- thank you.
cfset dayview = #createdate(year, month, thisday)#
cfquery name=ForToday dbtype=query
select *
from qEvents
where qEvents.TheDate = #dayview#
/cfquery
++
Kevin Parker
Advanced Imaging
e: webmas...@advancedimaging.com.au
w: www.advancedimaging.com.au
m: 0418 815 527
++
http://au.linkedin.com/in/krparker
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344811
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Baffing date problem
First id try something like:
Where qEvents.theDate = cfqueryparam cfsqltype=cf_sql_timestamp
value=#thisDay# /
If that doesn't work, Id try:
where qEvents.TheDate = cfqueryparam cfsqltype=cf_sql_timestamp
value=#createODBCDateTime(dayview)# /
If that doesn't work... read on.
If you are on SQL Server 2008, you could try something like...
where cast(qEvents.TheDate as date) = #createODBCDate(dayview)#
Other wise one of these might work...
Where cast(datePart(year, qEvents.TheDate) as nvarchar(4)) + '/' +
cast(datePart(month, qEvents.TheDate) as nvarchar(2)) + '/' +
cast(datePart(day, qEvents.TheDate) as nvarchar(2)) = #dateformat(dayview,
-mm-dd)#
Or maybe something goofy like...
Where qEvents.theDate = '#dayview# 00:00:00.0'
.:.:.:.:.:.:.:.:.:.:.:.:.:.
Bobby Hartsfield
http://acoderslife.com
http://cf4em.com
-Original Message-
From: Kevin Parker [mailto:tras...@internode.on.net]
Sent: Saturday, May 21, 2011 10:16 PM
To: cf-talk
Subject: Baffing date problem
I love CF but dates are definitely my Achilles heel.
I've dumped qEvents.TheDate and its formatted like {ts '2011-06-07
00:00:00.0'}
I've dumped dayview and its formatted like {ts '2011-06-07 00:00:00'}
I would have thought that as time stamps CF would see these the same despite
the 0 difference but the query returns no rows and I'm stumped. A cfdump of
qEvents shows the query is returning data correctly. Funny thing is this
works on a CF6 box running SQL2000 but fails on a CF8 box running SQL2005.
What I'm trying to do is do a check for qEvents that on a given day there
are or are not events for that day (thisday). Any advice greatly appreciated
- thank you.
cfset dayview = #createdate(year, month, thisday)#
cfquery name=ForToday dbtype=query
select *
from qEvents
where qEvents.TheDate = #dayview#
/cfquery
++
Kevin Parker
Advanced Imaging
e: webmas...@advancedimaging.com.au
w: www.advancedimaging.com.au
m: 0418 815 527
++
http://au.linkedin.com/in/krparker
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344812
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Baffing date problem
you need to make the same format on both sides of the SQL where clause
so try:
cfset dayview = #createODBCDate(createdate(year, month, thisday))#
On 22/05/2011 12:15, Kevin Parker wrote:
I love CF but dates are definitely my Achilles heel.
I've dumped qEvents.TheDate and its formatted like {ts '2011-06-07
00:00:00.0'}
I've dumped dayview and its formatted like {ts '2011-06-07 00:00:00'}
I would have thought that as time stamps CF would see these the same despite
the 0 difference but the query returns no rows and I'm stumped. A cfdump of
qEvents shows the query is returning data correctly. Funny thing is this
works on a CF6 box running SQL2000 but fails on a CF8 box running SQL2005.
What I'm trying to do is do a check for qEvents that on a given day there
are or are not events for that day (thisday). Any advice greatly appreciated
- thank you.
cfset dayview = #createdate(year, month, thisday)#
cfquery name=ForToday dbtype=query
select *
from qEvents
where qEvents.TheDate = #dayview#
/cfquery
--
Yours,
Kym Kovan
mbcomms.net.au
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344813
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Baffing date problem
Thanks Bobby (and the other good folk who chipped in too) - your first
option did the trick.
Where qEvents.theDate = cfqueryparam cfsqltype=cf_sql_timestamp
value=#thisDay#
++
Kevin Parker
Advanced Imaging
e: webmas...@advancedimaging.com.au
w: www.advancedimaging.com.au
m: 0418 815 527
++
http://au.linkedin.com/in/krparker
-Original Message-
From: Bobby Hartsfield [mailto:bo...@acoderslife.com]
Sent: Sunday, 22 May 2011 12:41
To: cf-talk
Subject: RE: Baffing date problem
First id try something like:
Where qEvents.theDate = cfqueryparam cfsqltype=cf_sql_timestamp
value=#thisDay# /
If that doesn't work, Id try:
where qEvents.TheDate = cfqueryparam cfsqltype=cf_sql_timestamp
value=#createODBCDateTime(dayview)# /
If that doesn't work... read on.
If you are on SQL Server 2008, you could try something like...
where cast(qEvents.TheDate as date) = #createODBCDate(dayview)#
Other wise one of these might work...
Where cast(datePart(year, qEvents.TheDate) as nvarchar(4)) + '/' +
cast(datePart(month, qEvents.TheDate) as nvarchar(2)) + '/' +
cast(datePart(day, qEvents.TheDate) as nvarchar(2)) = #dateformat(dayview,
-mm-dd)#
Or maybe something goofy like...
Where qEvents.theDate = '#dayview# 00:00:00.0'
.:.:.:.:.:.:.:.:.:.:.:.:.:.
Bobby Hartsfield
http://acoderslife.com
http://cf4em.com
-Original Message-
From: Kevin Parker [mailto:tras...@internode.on.net]
Sent: Saturday, May 21, 2011 10:16 PM
To: cf-talk
Subject: Baffing date problem
I love CF but dates are definitely my Achilles heel.
I've dumped qEvents.TheDate and its formatted like {ts '2011-06-07
00:00:00.0'}
I've dumped dayview and its formatted like {ts '2011-06-07 00:00:00'}
I would have thought that as time stamps CF would see these the same despite
the 0 difference but the query returns no rows and I'm stumped. A cfdump of
qEvents shows the query is returning data correctly. Funny thing is this
works on a CF6 box running SQL2000 but fails on a CF8 box running SQL2005.
What I'm trying to do is do a check for qEvents that on a given day there
are or are not events for that day (thisday). Any advice greatly appreciated
- thank you.
cfset dayview = #createdate(year, month, thisday)#
cfquery name=ForToday dbtype=query
select *
from qEvents
where qEvents.TheDate = #dayview#
/cfquery
++
Kevin Parker
Advanced Imaging
e: webmas...@advancedimaging.com.au
w: www.advancedimaging.com.au
m: 0418 815 527
++
http://au.linkedin.com/in/krparker
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:344814
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
