Re: JVM help!
Why do you believe this is a JVM problem? Jochem -- Jochem van Dieten http://jochem.vandieten.net/ ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342327 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: SELECT - Option selected
Does this not work on cfselect? If you are talking the HTML select element it is selected=selected Sent from my iPhone On Feb 15, 2011, at 4:47 PM, Torrent Girl moniqueb...@gmail.com wrote: ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342328 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: JVM help!
+1 on this suggestion... set a permsize.
Mark A. Kruger, MCSE, CFG
(402) 408-3733 ext 105
Skype: markakruger
www.cfwebtools.com
www.coldfusionmuse.com
www.necfug.com
-Original Message-
From: Carl Meyer [mailto:ca...@tassweb.com.au]
Sent: Wednesday, February 16, 2011 1:29 AM
To: cf-talk
Subject: Re: JVM help!
Hi Duncan, you do not have a -XX:PermSize=m set on your new 64 bit. Apply other
suggestions mentioned plus a minimum setting for Perminant Generation as
without one it can try size down to Java default. What size? The value depends
perhaps with the MaxPermSize you have set already use say half so
-XX:PermSize=512m .
You could also do some JVM logging to get an idea how the JVM args are running,
sometimes it is good to know rather than just guessing at it.
HTH, Carl.
Here is the setup on the old and new boxes:
4GB, Dual Xeon, 32bit windows, jdk1.6.0_13
java.args=-server -Xms512m -Xmx1000m -Dsun.io.useCanonCaches=false
-Dcoldfusion.fckupload=true -XX:PermSize=128m -XX:MaxPermSize=240m
-XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError
-Dsun.rmi.dgc.client.gcInterval=15
-Dsun.rmi.dgc.server.gcInterval=15
-Dcoldfusion.rootDir={application.home}/../
-Dcoldfusion.libPath={application.home}/../lib
-Dcoldfusion.classPath={application.home}/../lib/updates,{application.
home}/../lib,{application.home}/../gateway/lib/,{application.home}/..
/wwwroot/WEB-INF/flex/jars,{application.home}/..
/wwwroot/WEB-INF/cfform/jars
16Gb, Quad Core, 64bit Windows, jdk1.6.0_13
java.args=-server ââ¬âXmx8192m -Xms8192m -Dsun.io.useCanonCaches=false
-XX:MaxPermSize=1024m -XX:+UseParallelGC -Xbatch
-Dcoldfusion.rootDir={application.home}/
-Djava.security.policy={application.
home}/ servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/coldfusion.
policy
-Djava.security.auth.policy={application.
home }/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/neo_jaas.
policy
Runs great, has no crashes, happily inside memory limits, but runs so
slow.
Duncan I Loxton
duncan.lox...@gmail.com
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342329
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: JVM help!
Duncan, Yes... tell us what you mean by slow? Page execution is slow... Database queries... What's going on with the ap... What are these other aps... What leads you to believe it's the JVM? -Mark -Original Message- From: Jochem van Dieten [mailto:joch...@gmail.com] Sent: Wednesday, February 16, 2011 4:43 AM To: cf-talk Subject: Re: JVM help! Why do you believe this is a JVM problem? Jochem -- Jochem van Dieten http://jochem.vandieten.net/ ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342330 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: SELECT - Option selected
depends on cfselect... in simple cfselect - one based on an in-page query, for example - you can provide a value in the 'selected' attribute and an option with that value will be pre-selected. advanced cfselect - one with a binding - does not support this functionality. but there are javascript work-arounds for that: google 'cfselect binding selected' and you will find them Azadi On 16/02/2011 21:34 , Torrent Girl wrote: Does this not work on cfselect? If you are talking the HTML select element it is selected=selected Sent from my iPhone On Feb 15, 2011, at 4:47 PM, Torrent Girlmoniqueb...@gmail.com wrote: ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342331 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: JVM help!
Duncan,
Sorry about _MY_ confusion. I was reading and responding to your request, while
'out', with my phone and the small screen sometimes jumbles up a message enough
to be confusing. I think I saw this running out of memory and thought that
was the issue. I just noticed that the entire message didn't show up on my
phone. Sorry.
Still the JVM settings should be set as many have mentioned. However, the most
critical thing you can do is monitor everything. DB, CF, IIS, OS, network, name
resolution (DNS) just to name a few possible things. Is the DB on the same
machine? or on a network somewhere? make sure there is nothing wrong on the
network to slow things down. Is the new server connection to network resources
via IP addresses or using DNS? If using DNS, make sure name resolution happens
fast from the server. I actually seen slow name resolution (or timeouts of
resolution) cause issues on a server. There may not even be an issue with
ColdFusion or the JVM.
When I go to http://www.hairbodyskin.com.au/ my FF plugins tells me it took
10.449s from the time my http request was sent to the first response from the
server, and then a total of 15s to complete the response. (LORI plugin for FF).
Then sometimes the requests happen fast. This response times are all over the
spectrum here.
The other apps, are those heavily used? This is a new machine, but even so are
you overloading it with too many high load apps?
There are a lot of possible issues. Monitor everything.
Everyone including myself can offer advice on settings and potential causes,
but we are all blind to the real issue without accurate monitoring.
Wil Genovese
Sr. Web Application Developer/
Systems Administrator
CF Webtools
www.cfwebtools.com
wilg...@trunkful.com
www.trunkful.com
On Feb 16, 2011, at 1:27 AM, Duncan wrote:
Hi Will,
Thanks - but I am confused about your terminology. You state we are looking
for a memory issue, is this inextricably linked to the execution time? i.e.
could it be the JVM trying to cleanup the stuff in memory prior to the end
of the request?
As I pointed out the exact same code, with the exact same data works faster
on the old 32bit machine. If I can put some logic around this, on the
possible causes and how that relates to the JVM settings, I will perhaps be
able to make some better decisions.
Can you help expand on what we are looking for?
Thanks!
On Wed, Feb 16, 2011 at 4:09 PM, Wil Genovese wilg...@gmail.com wrote:
Set min max heap to be equal.
Set gc intervals to 1
And try again. These are not hard set numbers but in many cases works good
enough to start figuring out what is really happening.
You should install monitoring package and look for the cause of the memory
issue. Typically it's unexpected behavior in code. Many times a query
returning far more results than expected.
Monitor, fix code monitor again, adjust settings as needed.
Wil Genovese
Sr Web Application Developer / Sys Admin
CFWebtools
On Feb 15, 2011, at 10:48 PM, Duncan duncan.lox...@gmail.com wrote:
Hi Everyone,
We have recently put together a new CF server, moving from 32 bit, 4Gb
RAM
Windows over to 64bit, 16Gb RAM Windows.
We have pretty much done a default set up, and placed the same set of
applications on the new server, and have run in the machine for a bit.
After
a few weeks we have been noticing some cracks appear, and we now have
some
issues that we have narrowed down to the JVM/CF setup.
Here is the setup on the old and new boxes:
4GB, Dual Xeon, 32bit windows, jdk1.6.0_13
java.args=-server -Xms512m -Xmx1000m -Dsun.io.useCanonCaches=false
-Dcoldfusion.fckupload=true -XX:PermSize=128m -XX:MaxPermSize=240m
-XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError
-Dsun.rmi.dgc.client.gcInterval=15
-Dsun.rmi.dgc.server.gcInterval=15
-Dcoldfusion.rootDir={application.home}/../
-Dcoldfusion.libPath={application.home}/../lib
-Dcoldfusion.classPath={application.home}/../lib/updates,{application.home}/../lib,{application.home}/../gateway/lib/,{application.home}/../wwwroot/WEB-INF/flex/jars,{application.home}/../wwwroot/WEB-INF/cfform/jars
Crashed a lot, had out of memory issues once every hour on a good day.
www.hairbodyskin.com.au takes 2,781ms (from FusionReactor)
16Gb, Quad Core, 64bit Windows, jdk1.6.0_13
java.args=-server ââ¬âXmx8192m -Xms8192m -Dsun.io.useCanonCaches=false
-XX:MaxPermSize=1024m -XX:+UseParallelGC -Xbatch
-Dcoldfusion.rootDir={application.home}/
-Djava.security.policy={application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/coldfusion.policy
-Djava.security.auth.policy={application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/neo_jaas.policy
Runs great, has no crashes, happily inside memory limits, but runs so
slow.
www.hairbodyskin.com.au takes 10,361ms (from FusionReactor)
I must add
RE: JVM help!
When you get varied response times like that I often take a closer look at the
network (duplexing, flow control etc).
-Original Message-
From: Wil Genovese [mailto:jugg...@trunkful.com]
Sent: Wednesday, February 16, 2011 10:11 AM
To: cf-talk
Subject: Re: JVM help!
Duncan,
Sorry about _MY_ confusion. I was reading and responding to your request, while
'out', with my phone and the small screen sometimes jumbles up a message enough
to be confusing. I think I saw this running out of memory and thought that
was the issue. I just noticed that the entire message didn't show up on my
phone. Sorry.
Still the JVM settings should be set as many have mentioned. However, the most
critical thing you can do is monitor everything. DB, CF, IIS, OS, network, name
resolution (DNS) just to name a few possible things. Is the DB on the same
machine? or on a network somewhere? make sure there is nothing wrong on the
network to slow things down. Is the new server connection to network resources
via IP addresses or using DNS? If using DNS, make sure name resolution happens
fast from the server. I actually seen slow name resolution (or timeouts of
resolution) cause issues on a server. There may not even be an issue with
ColdFusion or the JVM.
When I go to http://www.hairbodyskin.com.au/ my FF plugins tells me it took
10.449s from the time my http request was sent to the first response from the
server, and then a total of 15s to complete the response. (LORI plugin for FF).
Then sometimes the requests happen fast. This response times are all over the
spectrum here.
The other apps, are those heavily used? This is a new machine, but even so are
you overloading it with too many high load apps?
There are a lot of possible issues. Monitor everything.
Everyone including myself can offer advice on settings and potential causes,
but we are all blind to the real issue without accurate monitoring.
Wil Genovese
Sr. Web Application Developer/
Systems Administrator
CF Webtools
www.cfwebtools.com
wilg...@trunkful.com
www.trunkful.com
On Feb 16, 2011, at 1:27 AM, Duncan wrote:
Hi Will,
Thanks - but I am confused about your terminology. You state we are looking
for a memory issue, is this inextricably linked to the execution time? i.e.
could it be the JVM trying to cleanup the stuff in memory prior to the end
of the request?
As I pointed out the exact same code, with the exact same data works faster
on the old 32bit machine. If I can put some logic around this, on the
possible causes and how that relates to the JVM settings, I will perhaps be
able to make some better decisions.
Can you help expand on what we are looking for?
Thanks!
On Wed, Feb 16, 2011 at 4:09 PM, Wil Genovese wilg...@gmail.com wrote:
Set min max heap to be equal.
Set gc intervals to 1
And try again. These are not hard set numbers but in many cases works good
enough to start figuring out what is really happening.
You should install monitoring package and look for the cause of the memory
issue. Typically it's unexpected behavior in code. Many times a query
returning far more results than expected.
Monitor, fix code monitor again, adjust settings as needed.
Wil Genovese
Sr Web Application Developer / Sys Admin
CFWebtools
On Feb 15, 2011, at 10:48 PM, Duncan duncan.lox...@gmail.com wrote:
Hi Everyone,
We have recently put together a new CF server, moving from 32 bit, 4Gb
RAM
Windows over to 64bit, 16Gb RAM Windows.
We have pretty much done a default set up, and placed the same set of
applications on the new server, and have run in the machine for a bit.
After
a few weeks we have been noticing some cracks appear, and we now have
some
issues that we have narrowed down to the JVM/CF setup.
Here is the setup on the old and new boxes:
4GB, Dual Xeon, 32bit windows, jdk1.6.0_13
java.args=-server -Xms512m -Xmx1000m -Dsun.io.useCanonCaches=false
-Dcoldfusion.fckupload=true -XX:PermSize=128m -XX:MaxPermSize=240m
-XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError
-Dsun.rmi.dgc.client.gcInterval=15
-Dsun.rmi.dgc.server.gcInterval=15
-Dcoldfusion.rootDir={application.home}/../
-Dcoldfusion.libPath={application.home}/../lib
-Dcoldfusion.classPath={application.home}/../lib/updates,{application.home}/../lib,{application.home}/../gateway/lib/,{application.home}/../wwwroot/WEB-INF/flex/jars,{application.home}/../wwwroot/WEB-INF/cfform/jars
Crashed a lot, had out of memory issues once every hour on a good day.
www.hairbodyskin.com.au takes 2,781ms (from FusionReactor)
16Gb, Quad Core, 64bit Windows, jdk1.6.0_13
java.args=-server âââ‰â¬ÅXmx8192m -Xms8192m
-Dsun.io.useCanonCaches=false
-XX:MaxPermSize=1024m -XX:+UseParallelGC -Xbatch
-Dcoldfusion.rootDir={application.home}/
cfquery returning 0 in scientific notation in CF 8 but not CF 7
Greetings. In one of our systems we're upgrading from ColdFusion 7 to CF 8. While testing a piece of functionality we were getting 0 returned as 0E-8 for a query using a stored procedure. If the value is greater than 0, there's no issue. Updating the procedure to cast the returned value as a numeric resolved the issue, but since this functionality is working fine against CF 7 ... We're using MS SQL, and the data source settings in ColdFusion Administrator are consistent between 7 and 8. Google and searching these forums isn't helping much, other than suggesting it's coming out of MSSQL, but ... does anyone have any additional background on why we're seeing this? Do we need to evaluate each of our procedures/calls to verify that values are coming back correctly and cast if not, or ...? Thanks! ~James ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342334 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: cfquery returning 0 in scientific notation in CF 8 but not CF 7
What is the data type of the field in SQL Server? If you run the query in Management Studio, what do you see for that value? How are you outputting the value in CF to see the 0E-8 number? What tag are you using to call the stored procedure in ColdFusion? Which version of MS SQL Server are you using? -Mike Chabot On Wed, Feb 16, 2011 at 11:20 AM, James Skemp jsk...@wisbar.org wrote: Greetings. In one of our systems we're upgrading from ColdFusion 7 to CF 8. While testing a piece of functionality we were getting 0 returned as 0E-8 for a query using a stored procedure. If the value is greater than 0, there's no issue. Updating the procedure to cast the returned value as a numeric resolved the issue, but since this functionality is working fine against CF 7 ... We're using MS SQL, and the data source settings in ColdFusion Administrator are consistent between 7 and 8. Google and searching these forums isn't helping much, other than suggesting it's coming out of MSSQL, but ... does anyone have any additional background on why we're seeing this? Do we need to evaluate each of our procedures/calls to verify that values are coming back correctly and cast if not, or ...? Thanks! ~James ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342335 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
Hey everybody, we're in a bind here and I pitched the idea to the client of bringing in a ringer. Is there someone out there who is interested in building a direct-to-SQL Server or direct-to-mySQL-based solution? Solution would have to, on a scheduled basis, grab a file in a location on the server on its own without CF and process it (we have stuff that retrieves/places the file). Current file size is about 50 mb but must be expected to go to around 100. At 42mb using just CF I am at about 70 seconds per record loop. We need a *significant* improvement in performance. Work would be on a dedicated Windows server. Present box is a fairly powerful Crystaltech Win2003 Server w/4GB RAM and a Xeon 2.8 processor w/4 cores. We'd consider a Win2008, 64-bit installation to improve speed. Looking at a 24-core server at Viviotech w/16gb as a hi-horsepower alternative. Our feed partner is probably going to be able to use a different method of delivery that will reduce file size, but for now we need to plan for the worst and move on it. Anyone interested? You can email me at my for-reals email at matt AT mysecretbase DOT com. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342336 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: cfquery returning 0 in scientific notation in CF 8 but not CF 7
Thanks for the clarification questions, Mike. What is the data type of the field in SQL Server? numeric(10,2) If you run the query in Management Studio, what do you see for that value? 0.00 How are you outputting the value in CF to see the 0E-8 number? In a tossed error it was displayed in the stack trace and a cfdump of the returned information from the initial cfquery call against the sp showed it there as well. When it works, it's stored to a variable before being passed to SQL via another query. What tag are you using to call the stored procedure in ColdFusion? cfquery Which version of MS SQL Server are you using? 64 bit, 2005 Standard edition SP3 ~James ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342337 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
Hi, Matt... I know you might have looked at this link at dev.mysql.com for a solution to processing your data feed, but I thought I'd bring it up, just in case you haven't seen it. Looks like it might be right up your alley. http://dev.mysql.com/tech-resources/articles/xml-in-mysql5.1-6.0.html#xml-5. 1-in-and-out While I didn't have to process xml files, I do have to process a daily data feed of real estate data in comma-delimited files and insert that info into a MySQL 5 database. At first, I tried processing the files with CF, but it was a *really* slow process using CF, taking almost a minute to process some files. Finally, someone on this list suggested I take a look at the MySQL commands for loading data from files, such as, in your case, Load_File(), which opens an entire XML document, makes it available as a string, and inserts this string into a table column. In my case, using MySQL's load data infile, the time was reduced to less than a second. Check out the page above for details on usage and syntax. hth, Rick -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Wednesday, February 16, 2011 12:22 PM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? Hey everybody, we're in a bind here and I pitched the idea to the client of bringing in a ringer. Is there someone out there who is interested in building a direct-to-SQL Server or direct-to-mySQL-based solution? Solution would have to, on a scheduled basis, grab a file in a location on the server on its own without CF and process it (we have stuff that retrieves/places the file). Current file size is about 50 mb but must be expected to go to around 100. At 42mb using just CF I am at about 70 seconds per record loop. We need a *significant* improvement in performance. Work would be on a dedicated Windows server. Present box is a fairly powerful Crystaltech Win2003 Server w/4GB RAM and a Xeon 2.8 processor w/4 cores. We'd consider a Win2008, 64-bit installation to improve speed. Looking at a 24-core server at Viviotech w/16gb as a hi-horsepower alternative. Our feed partner is probably going to be able to use a different method of delivery that will reduce file size, but for now we need to plan for the worst and move on it. Anyone interested? You can email me at my for-reals email at matt AT mysecretbase DOT com. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342338 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: cfquery returning 0 in scientific notation in CF 8 but not CF 7
It occurred to me that I should first ask if this issue is causing any problems, or if it is only an unexpected way to format a number? For example 0.00 = 0E-8. They are both the same number regardless of how they are displayed. For clarification, which number has the unexpected formatting, the value pulled directly out of a database table and placed in the cfquery record set, the variable that is being assigned the value, or both? -Mike Chabot On Wed, Feb 16, 2011 at 12:29 PM, James Skemp jsk...@wisbar.org wrote: Thanks for the clarification questions, Mike. What is the data type of the field in SQL Server? numeric(10,2) If you run the query in Management Studio, what do you see for that value? 0.00 How are you outputting the value in CF to see the 0E-8 number? In a tossed error it was displayed in the stack trace and a cfdump of the returned information from the initial cfquery call against the sp showed it there as well. When it works, it's stored to a variable before being passed to SQL via another query. What tag are you using to call the stored procedure in ColdFusion? cfquery Which version of MS SQL Server are you using? 64 bit, 2005 Standard edition SP3 ~James ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342339 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: cfx_pwtextcrypt CF9
Thanks for your help, I was able to get this working using your suggestions and the original code from encryption http://www.12robots.com/index.cfm/2010/7/19/Using-Asymmetric-Cryptography-in -your-ColdFusion-Application--Security-Series-1610. I modified that code to add support for the following: 1. Wrapped it all up in a nice CFC 2. Added a generateKeyPair() method to generate the keys 3. Added support for returning the keys as base64 encoded strings 4. The encrypt/decrypt methods accept either a string, java instance or binary object for the public/private key pair. If a string or binary object is provided, it will create the java instance from the string/binary argument and then do the decryption. I needed it to work this way, so I could save the public key to the database and save the private key to a text file (which is downloaded by our end user for them to hold on to). I submitted the CFC to RIAForge in case any one else might find it useful. I believe I am just waiting for approval now. If any one wants a copy just email me and I'll send it to you. Brook Davies -Original Message- From: denstar [mailto:valliants...@gmail.com] Sent: February-15-11 7:31 PM To: cf-talk Subject: Re: cfx_pwtextcrypt CF9 There are examples of doing lots of weird PGP related things here: http://subversion.assembla.com/svn/cfmlprojects/trunk/src/cfopenpgp/src/tag/ cfopenpgp/cfc/openpgp.cfc This uses BouncyCastle (you have to add the jars), so it's sorta stand-alone from what CF comes with. It's butt-ugly, but there are some tests a couple dirs up and over... Sounds like the original question was sorted, but I'll toss it out there anyway, as there's examples in there of doing most the java-ish conversions and whatnot (bytearray stuff, etc.). :Den -- Ideas any one can mould as he wishes. Josiah Royce On Thu, Feb 10, 2011 at 1:53 PM, Brook Davies wrote: We've been using cfx_pwtextcrypt since 2002, its worked fine on all CF versions right up to CF8. However, it looks like on the 64 Bit CF9, it no longer works. I assume it's a COM related issue. Does anyone know for sure? The tag was able to generate a public/private key pair using RSA 512 bit encryption and subsequently perform encryption/decryption using the keys generated. Does anyone know if there is a way to do this with CF9 without this CFX? Brook D. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342340 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: cfquery returning 0 in scientific notation in CF 8 but not CF 7
It occurred to me that I should first ask if this issue is causing any problems, or if it is only an unexpected way to format a number? For example 0.00 = 0E-8. They are both the same number regardless of how they are displayed. Unfortunately, it is causing an issue in this particular instance (and is in fact why we had a stack trace to see that 0E-8 was being returned) :) It's legacy code, so with some work we could certainly resolve the issue, but, as a legacy *third-party* solution, my concern is it's going to cause issues in other parts of the system. Assuming this is expected behavior in CF8/we haven't flipped the appropriate 'switch.' For clarification, which number has the unexpected formatting, the value pulled directly out of a database table and placed in the cfquery record set, the variable that is being assigned the value, or both? Probably the former - ColdFusion's interpretation of the value from SQL / the value in the cfquery record set, since the resolution in this particular case is to cast the returned value to a particular type in the procedure. But, another way to resolve the issue is to check the formatting when setting the variable, so I suppose one could argue both. The immediate problem is because of the latter, but the root cause is the former. In short, 0E-8 is populated in the proc's cfquery result set, a variable is being set with 0E-8, and another query is failing because it doesn't care for 0E-8 being passed. Hopefully that last bit didn't muddy the waters ... :) ~James ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342341 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
Yeah I have seen that, but I'm up against a time constraint and there are a number of issues I don't have my head wrapped around, and the client needs a solution asap given that he has a daily feed encompassing thousands of records and dozens of clients ... that has now been dead about 10 days. I'm sure I could figure it out with enough time, but the client would be better served if someone who actually has done this can hit the ground running on it. BUT with this whole thing in mind, there are some things I don't get about this whole load_file() thing that maybe you or someone else can answer. I get how the file is pulled into memory, but how can a multi-level file get pulled in? Hopefully this excerpt (and its only a fragment) will explain my problem. - I don't need the header sectioon and want to skip over that. - The dataArea contains the line item records I want to pull in. - I can write CF code to pull the price and currency out of the pricing section, but how would mySQL do something like that where the xml design was made as complicated as humanly possible? (this format is StarStandard.org-based, btw) - Note the ImageAttachmentExtended records. There can be any number of those, or none. A CFloop works great but ... - the VehicleRemarketingBoat section can have any number of line items *or groups* and they vary from record to record. The groups can have more than one entry in them, or not. Here again CF gives me tools to deal with this. I just don't see how something like this can be dumped into a db at all. I'm assuming its inexperience with the task. Or is this xml a bit beyond that type of approach? headerSection headerField1blah/headerField1 headerField2blah/headerField2 headerField3yoo/headerField3 headerField4hoo/headerField4 headerField5woof/headerField5 /headerSection dataArea dataRecord recordID123/recordID year1980/year makeTrabant/make modelTurboXL/model pricing hidePricefalse/hidePrice hidePrice8/hidePrice Price ChargeAmount currencyID='USD'99500/ChargeAmount /Price /pricing Location LocationAddress CityNameAnnapolis/CityName CountryIDUS/CountryID StateOrProvinceCountrySubDivisionIDMD/StateOrProvinceCountrySubDivisionID /LocationAddress /Location ImageAttachmentExtended ID46795014/ID URIhttp://img.arf.com/images/1/70/33/3197033_0_08022024_1.jpg/URI ImageWidthMeasure unitCode='pixel'412/ImageWidthMeasure ImageHeightMeasure unitCode='pixel'471/ImageHeightMeasure ImageLastModifiedDateTime2011-02-08T11:24:00-08:00/ImageLastModifiedDateTime UsagePreference PriorityRankingNumeric1/PriorityRankingNumeric /UsagePreference /ImageAttachmentExtended ImageAttachmentExtended ID123456/ID URIhttp://img.arf.com/images/1/70/33/3197033_0_08022024_2.jpg/URI ImageWidthMeasure unitCode='pixel'412/ImageWidthMeasure ImageHeightMeasure unitCode='pixel'471/ImageHeightMeasure ImageLastModifiedDateTime2011-02-08T11:24:00-08:00/ImageLastModifiedDateTime UsagePreference PriorityRankingNumeric2/PriorityRankingNumeric /UsagePreference /ImageAttachmentExtended ImageAttachmentExtended ID987654/ID URIhttp://img.arf.com/images/1/70/33/3197033_0_08022024_3.jpg/URI ImageWidthMeasure unitCode='pixel'412/ImageWidthMeasure ImageHeightMeasure unitCode='pixel'471/ImageHeightMeasure ImageLastModifiedDateTime2011-02-08T11:24:00-08:00/ImageLastModifiedDateTime UsagePreference PriorityRankingNumeric1/PriorityRankingNumeric /UsagePreference /ImageAttachmentExtended VehicleRemarketingBoat MakeStringStarratt and Jenks/MakeString ModelYear1979/ModelYear SaleClassCodeUsed/SaleClassCode ModelMorgan 45/Model BoatLengthGroup BoatLengthCodeNominal Length/BoatLengthCode BoatLengthMeasure unitCode='feet'45/BoatLengthMeasure /BoatLengthGroup BoatLengthGroup BoatLengthCodeLength At Water Line/BoatLengthCode BoatLengthMeasure unitCode='feet'31.42/BoatLengthMeasure /BoatLengthGroup BoatLengthGroup BoatLengthCodeLength Overall/BoatLengthCode BoatLengthMeasure unitCode='feet'45/BoatLengthMeasure /BoatLengthGroup BeamMeasure unitCode='feet'11/BeamMeasure DraftMeasureGroup DraftMeasure unitCode='feet'6.08/DraftMeasure BoatDraftCodeMax Draft/BoatDraftCode /DraftMeasureGroup
Re: Big XML files processing Really s-l-o-w. Solution?
Matt, as you want a non CF solution, you might be best posting elsewhere to find a database guru who could do it all directly on the DB. If no-one here pipes up then try www.odesk.com On Wed, Feb 16, 2011 at 6:09 PM, Rick Faircloth r...@whitestonemedia.comwrote: Hi, Matt... I know you might have looked at this link at dev.mysql.com for a solution to processing your data feed, but I thought I'd bring it up, just in case you haven't seen it. Looks like it might be right up your alley. http://dev.mysql.com/tech-resources/articles/xml-in-mysql5.1-6.0.html#xml-5. 1-in-and-out While I didn't have to process xml files, I do have to process a daily data feed of real estate data in comma-delimited files and insert that info into a MySQL 5 database. At first, I tried processing the files with CF, but it was a *really* slow process using CF, taking almost a minute to process some files. Finally, someone on this list suggested I take a look at the MySQL commands for loading data from files, such as, in your case, Load_File(), which opens an entire XML document, makes it available as a string, and inserts this string into a table column. In my case, using MySQL's load data infile, the time was reduced to less than a second. Check out the page above for details on usage and syntax. hth, Rick -Original Message- From: Matt Robertson [mailto:websitema...@gmail.com] Sent: Wednesday, February 16, 2011 12:22 PM To: cf-talk Subject: Re: Big XML files processing Really s-l-o-w. Solution? Hey everybody, we're in a bind here and I pitched the idea to the client of bringing in a ringer. Is there someone out there who is interested in building a direct-to-SQL Server or direct-to-mySQL-based solution? Solution would have to, on a scheduled basis, grab a file in a location on the server on its own without CF and process it (we have stuff that retrieves/places the file). Current file size is about 50 mb but must be expected to go to around 100. At 42mb using just CF I am at about 70 seconds per record loop. We need a *significant* improvement in performance. Work would be on a dedicated Windows server. Present box is a fairly powerful Crystaltech Win2003 Server w/4GB RAM and a Xeon 2.8 processor w/4 cores. We'd consider a Win2008, 64-bit installation to improve speed. Looking at a 24-core server at Viviotech w/16gb as a hi-horsepower alternative. Our feed partner is probably going to be able to use a different method of delivery that will reduce file size, but for now we need to plan for the worst and move on it. Anyone interested? You can email me at my for-reals email at matt AT mysecretbase DOT com. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342343 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
(ot) field data restriction
I have some Javascript code below that restricts a user from typing in letters,
but they can type in numbers.
Is there a way I can allow one character that is not a number? How can I edit
the code to allow the : character only as well as numbers?
Thanks.
RO
HWW
SCRIPT LANGUAGE=JavaScript
function checkIt(evt) {
evt = (evt) ? evt : window.event
var charCode = (evt.which) ? evt.which : evt.keyCode
if (charCode 31 (charCode 48 || charCode 57)) {
status = This field accepts numbers only.
return false
}
status =
return true
}
/SCRIPT
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342344
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Inline structure notation as argument fails
I'm re-writing an event handler to utilize CF9s scripting enhancement. The majority of the handler was already written in cfscript, including the line of code I'm having problems with. After making the change to full script, I'm getting an exception indicating Variable EVENT is undefined. http://coldbox.pastebin.com/CGRVf25z By moving the structure to its own variable, the code executes without error. Is this expected behavior? Jason Durham ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342345 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: CFX_HTTP5 - returns partial string
Thanks for your reply. In this case CFHTTP won't do it since the connection requires NTLM due to internal security requirements. OK. In that case, you might consider using Apache Commons HTTPClient instead. This is a Java library (or set of libraries) that will definitely work from a 64-bit environment. Here's a post on using this: http://www.terrenceryan.com/blog/post.cfm/digest-authentication-in-coldfusion On a side note, we might need some Google mini training out here for one of our server guys. Who should I contact at your office Dave? For training, you can send directly to train...@figleaf.com. Or you can send it to me, and I'll send it along - whichever works best. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342346 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: JVM help!
Hi everyone,
thanks for all the good feedback.
Let me first tell you WHY I think its a JVM issue, then show you what
results I have had.
We took the exact same code, and connected it to the exact same DB (and DB
server) and ran the exact same code in isolation. The comparison was
outstanding about 2s on the old machine and 10s as you have seen on the new.
The connection between the SQL and both web machines pinged at 1ms
We tried the connection from various locations and ruled out network issues
(although there could be ). We looked at execution times and logged the
information pretty heavily in the intensive parts of the app and saw that,
on comparison between machines the differences. Given that, and the
isolation tests, the only thing we could see that was different was the JVM
settings, thus the conclusion we came too.
so having taken some of the suggestions into account I have ended up with
this setting:
java.args=-server -Xms4096m -Xmx8192m -Dsun.io.useCanonCaches=false
-XX:PermSize=1024m -XX:MaxPermSize=1024m
-Dsun.rmi.dgc.client.gcInterval=15
-Dsun.rmi.dgc.server.gcInterval=15 -XX:+UseParallelGC -Xbatch
-Dcoldfusion.rootDir={application.home}/
-Djava.security.policy={application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/coldfusion.policy
-Djava.security.auth.policy={application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/neo_jaas.policy
This puts us at an execution time parallel to the old 32 bit machine, down
at 1.5-2 seconds. while this is a vast improvement I would hope that more
gains could be had out of tuning the JVM some more. I will tweak the
GCinterval and the permSize tomorrow night and see if there is an
incremental gain to be had.
Any other suggestions would be great!
Thanks.
On Thu, Feb 17, 2011 at 3:20 AM, Mark A. Kruger mkru...@cfwebtools.comwrote:
When you get varied response times like that I often take a closer look at
the network (duplexing, flow control etc).
-Original Message-
From: Wil Genovese [mailto:jugg...@trunkful.com]
Sent: Wednesday, February 16, 2011 10:11 AM
To: cf-talk
Subject: Re: JVM help!
Duncan,
Sorry about _MY_ confusion. I was reading and responding to your request,
while 'out', with my phone and the small screen sometimes jumbles up a
message enough to be confusing. I think I saw this running out of memory
and thought that was the issue. I just noticed that the entire message
didn't show up on my phone. Sorry.
Still the JVM settings should be set as many have mentioned. However, the
most critical thing you can do is monitor everything. DB, CF, IIS, OS,
network, name resolution (DNS) just to name a few possible things. Is the
DB on the same machine? or on a network somewhere? make sure there is
nothing wrong on the network to slow things down. Is the new server
connection to network resources via IP addresses or using DNS? If using DNS,
make sure name resolution happens fast from the server. I actually seen slow
name resolution (or timeouts of resolution) cause issues on a server. There
may not even be an issue with ColdFusion or the JVM.
When I go to http://www.hairbodyskin.com.au/ my FF plugins tells me it
took 10.449s from the time my http request was sent to the first response
from the server, and then a total of 15s to complete the response. (LORI
plugin for FF). Then sometimes the requests happen fast. This response
times are all over the spectrum here.
The other apps, are those heavily used? This is a new machine, but even so
are you overloading it with too many high load apps?
There are a lot of possible issues. Monitor everything.
Everyone including myself can offer advice on settings and potential
causes, but we are all blind to the real issue without accurate monitoring.
Wil Genovese
Sr. Web Application Developer/
Systems Administrator
CF Webtools
www.cfwebtools.com
wilg...@trunkful.com
www.trunkful.com
On Feb 16, 2011, at 1:27 AM, Duncan wrote:
Hi Will,
Thanks - but I am confused about your terminology. You state we are
looking
for a memory issue, is this inextricably linked to the execution time?
i.e.
could it be the JVM trying to cleanup the stuff in memory prior to the
end
of the request?
As I pointed out the exact same code, with the exact same data works
faster
on the old 32bit machine. If I can put some logic around this, on the
possible causes and how that relates to the JVM settings, I will perhaps
be
able to make some better decisions.
Can you help expand on what we are looking for?
Thanks!
On Wed, Feb 16, 2011 at 4:09 PM, Wil Genovese wilg...@gmail.com wrote:
Set min max heap to be equal.
Set gc intervals to 1
And try again. These are not hard set numbers but in many cases works
good
enough to start figuring out what is really happening.
You should install monitoring package and look for the cause of the
memory
issue. Typically it's
Re: cfquery returning 0 in scientific notation in CF 8 but not CF 7
Why would a SQL cast fix the issue? Casting a numeric(10,2) to a numeric(10,2) would not solve anything, which means that the value you are casting to numeric must be of a different data type, likely float or varchar. Floating point numbers often display in scientific notation. The numeric value is possibly being automatically converted into a floating point number by its use in a math function. It might help to see the SQL with the cast function that is fixing the issue. SELECT numeric_col FROM table should not require any further casting and you would not see the problem you are seeing. However, if you do something like this SELECT '000' + numeric_col + 1.2E-20 / 1.234 + 'cat' - 'dog' as newval FROM table then you are no longer dealing with a numeric, even though the source number might have started out as one. I'm still not clear on what the problem is. As a reminder, try to create a simplified example that isolates the problem being seen, away from any surrounding code. During the exercise of creating the simplified example, testing each increasingly complex iteration of the code, the source of the error will likely be revealed. If you were hoping for a quick answer to your question, I don't have one. Perhaps someone else has seen this issue when upgrading CF. -Mike Chabot On Wed, Feb 16, 2011 at 1:43 PM, James Skemp jsk...@wisbar.org wrote: It occurred to me that I should first ask if this issue is causing any problems, or if it is only an unexpected way to format a number? For example 0.00 = 0E-8. They are both the same number regardless of how they are displayed. Unfortunately, it is causing an issue in this particular instance (and is in fact why we had a stack trace to see that 0E-8 was being returned) :) It's legacy code, so with some work we could certainly resolve the issue, but, as a legacy *third-party* solution, my concern is it's going to cause issues in other parts of the system. Assuming this is expected behavior in CF8/we haven't flipped the appropriate 'switch.' For clarification, which number has the unexpected formatting, the value pulled directly out of a database table and placed in the cfquery record set, the variable that is being assigned the value, or both? Probably the former - ColdFusion's interpretation of the value from SQL / the value in the cfquery record set, since the resolution in this particular case is to cast the returned value to a particular type in the procedure. But, another way to resolve the issue is to check the formatting when setting the variable, so I suppose one could argue both. The immediate problem is because of the latter, but the root cause is the former. In short, 0E-8 is populated in the proc's cfquery result set, a variable is being set with 0E-8, and another query is failing because it doesn't care for 0E-8 being passed. Hopefully that last bit didn't muddy the waters ... :) ~James ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342348 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: cfquery returning 0 in scientific notation in CF 8 but not CF 7
Thanks Mike.
Opened up the procedure and gave it a look. It looks like there's some
multiplication and division going on, which is resulting in a change. Relevant
part for one of the returned values:
MemberPrice =
(case when qp.productid is null then p.MemberPrice
else
case when qp.MemberPrice is null then
p.MemberPrice*((100-qp.memberdiscountpercent)/100)
else qp.MemberPrice
end
end)
Running this query on ColdFusion 7.('current') against MS SQL returns (again,
cfdump of the cfquery, and the same thing as SQL Studio returns):
0.
On CF 8, we instead get:
0E-8
So, updated CF 8 SQL Driver perhaps? Either way, unless there's an
Administrator change, this does mean we'll have to evaluate code since, right
or wrong, CF 7 doesn't make this change.
Thanks!
~James
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342349
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: cfquery returning 0 in scientific notation in CF 8 but not CF 7
Looks like you might need to revise the procedure to CAST the calculated
value back to Numeric(10,2) before returning it?
Carl
On 2/16/2011 2:30 PM, James Skemp wrote:
Thanks Mike.
Opened up the procedure and gave it a look. It looks like there's some
multiplication and division going on, which is resulting in a change.
Relevant part for one of the returned values:
MemberPrice =
(case when qp.productid is null then p.MemberPrice
else
case when qp.MemberPrice is null then
p.MemberPrice*((100-qp.memberdiscountpercent)/100)
else qp.MemberPrice
end
end)
Running this query on ColdFusion 7.('current') against MS SQL returns (again,
cfdump of the cfquery, and the same thing as SQL Studio returns):
0.
On CF 8, we instead get:
0E-8
So, updated CF 8 SQL Driver perhaps? Either way, unless there's an
Administrator change, this does mean we'll have to evaluate code since, right
or wrong, CF 7 doesn't make this change.
Thanks!
~James
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342350
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: JVM help!
On Wed, Feb 16, 2011 at 1:34 PM, Duncan duncan.lox...@gmail.com wrote: java.args=-server -Xms4096m -Xmx8192m -Dsun.io.useCanonCaches=false One thing I would be cautious about here: a heap that large can be susceptible to stop the world GC sweeps and if the heap ever grows near the 8GB max, those sweeps will take a significant amount of time. What you'll see is generally really, really good response times and then every now and then a complete freeze of your app for all users for 10-30 seconds, then back to really, really good response times. In general, for long running processes, I'd try to keep the heap between 2GB and 3GB and instead run multiple instances and distribute load that way instead. Also, the PermGen area is mostly for .class files so it needs to be large enough to hold every class your system is going to need but once it hits that, it shouldn't grow (unless you're running code that creates new class loaders and loads classes repeatedly, e.g., JavaLoader when it's not cached in server scope etc). I've never needed a PermGen size as big a 1GB but you might have many tens of thousands of .class files (either lots of files or lots of methods in CFCs). However, if your settings are working for now, that's good. JVM tuning tends to be an ongoing thing - and somewhat of a black art :( -- Sean A Corfield -- (904) 302-SEAN Railo Technologies, Inc. -- http://getrailo.com/ An Architect's View -- http://corfield.org/ If you're not annoying somebody, you're not really alive. -- Margaret Atwood ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342351 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Inline structure notation as argument fails
CF9 or CF9.0.1? ISTR a bug in this area that got fixed sometime along the way...? On Wed, Feb 16, 2011 at 1:05 PM, Jason Durham jqdur...@gmail.com wrote: I'm re-writing an event handler to utilize CF9s scripting enhancement. The majority of the handler was already written in cfscript, including the line of code I'm having problems with. After making the change to full script, I'm getting an exception indicating Variable EVENT is undefined. http://coldbox.pastebin.com/CGRVf25z By moving the structure to its own variable, the code executes without error. Is this expected behavi ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342352 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Mon, Feb 14, 2011 at 11:44 PM, Matt Robertson wrote: I am tasked with doing an import on a file whose size can essentially be unlimited. We've been able to handle in the 10-15mb range but it recently ballooned to 100 mb, and its going to get larger. Processing time seems to be about 66 hours for a 45 mb file and thats a disaster. Would it help if you split the file into individual records before processing them? http://jochem.vandieten.net/2011/02/17/splitting-large-xml-files-with-coldfusion-and-stax/ Jochem -- Jochem van Dieten http://jochem.vandieten.net/ ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342353 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: cfquery returning 0 in scientific notation in CF 8 but not CF 7
Looks like you might need to revise the procedure to CAST the calculated value back to Numeric(10,2) before returning it? Carl Yeah, which is what we're doing for this one to resolve the issue. But since this is a legacy, third-party, application ... it worries me the number of places this may occur. And of course, as we all know, there's usually one place that gets missed in these types of situations ... :D I was hoping it was a configuration option that we missed, but ... I don't suppose someone with a similar environment could test against CF 9, could they? Thanks Carl! ~James ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342354 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Using Google Apps/Gmail for CFMail - limitations?
I'm switching several domains over from multiple shared hosting accounts to a virtual private server. For a number of reasons (server resources, managing accounts), I am using the Gmail that comes with Google Apps for domain email for each domain.Without an SMTP server, I'll need to use GMail for mail sending. I've found references to others doing that here, but I'm wondering what the limits are for sending email? Does Google throttle or otherwise inhibit email? One of the domains has about 5k users in an opt-in email list, and I want to make sure I'm not going to cause myself problems. Emails are sent from this site once, perhaps twice per week, so I think I'm looking at 10,000 outgoing emails weekly. Thanks, Pete ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342355 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Using Google Apps/Gmail for CFMail - limitations?
In my personal experience, Gmail for Domains is perfectly fine for sending/receiving small bits of mail here and there. However, with a mailing list of only about 300, it fails miserably. Very few go through, and it is quite arbitrary as to which ones go and which ones don't. That said, it is my understanding that if you are using the paid version, then you're okay. But you can definitely count on the free service *not* meeting your needs. HTH ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342356 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Using Google Apps/Gmail for CFMail - limitations?
OK, using GMail is probably out, then. What do I need to install/enable in order to send mail from multiple domains on a Windows 2008 Server? Just install MS SMTP, or are there better (free) third-party SMTP servers? Thanks, Pete On Wed, Feb 16, 2011 at 7:59 PM, Matt Quackenbush quackfu...@gmail.comwrote: In my personal experience, Gmail for Domains is perfectly fine for sending/receiving small bits of mail here and there. However, with a mailing list of only about 300, it fails miserably. Very few go through, and it is quite arbitrary as to which ones go and which ones don't. That said, it is my understanding that if you are using the paid version, then you're okay. But you can definitely count on the free service *not* meeting your needs. HTH ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342357 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Using Google Apps/Gmail for CFMail - limitations?
I think you should serious consider something like a PowerMTA server at your hosting company... On Wed, Feb 16, 2011 at 4:51 PM, Pete Ruckelshaus pruckelsh...@gmail.com wrote: I'm switching several domains over from multiple shared hosting accounts to a virtual private server. For a number of reasons (server resources, managing accounts), I am using the Gmail that comes with Google Apps for domain email for each domain. Without an SMTP server, I'll need to use GMail for mail sending. I've found references to others doing that here, but I'm wondering what the limits are for sending email? Does Google throttle or otherwise inhibit email? One of the domains has about 5k users in an opt-in email list, and I want to make sure I'm not going to cause myself problems. Emails are sent from this site once, perhaps twice per week, so I think I'm looking at 10,000 outgoing emails ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342358 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Using Google Apps/Gmail for CFMail - limitations?
I think that'd be overkill, given the relatively low volume. FWIW, I'm with KickAssVPS and have been happy with them now that I have my server set up. Pete On Wed, Feb 16, 2011 at 9:01 PM, Sean Corfield seancorfi...@gmail.comwrote: I think you should serious consider something like a PowerMTA server at your hosting company... On Wed, Feb 16, 2011 at 4:51 PM, Pete Ruckelshaus pruckelsh...@gmail.com wrote: I'm switching several domains over from multiple shared hosting accounts to a virtual private server. For a number of reasons (server resources, managing accounts), I am using the Gmail that comes with Google Apps for domain email for each domain.Without an SMTP server, I'll need to use GMail for mail sending. I've found references to others doing that here, but I'm wondering what the limits are for sending email? Does Google throttle or otherwise inhibit email? One of the domains has about 5k users in an opt-in email list, and I want to make sure I'm not going to cause myself problems. Emails are sent from this site once, perhaps twice per week, so I think I'm looking at 10,000 outgoing emails ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342359 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Big XML files processing Really s-l-o-w. Solution?
On Wed, Feb 16, 2011 at 3:49 PM, Jochem van Dieten wrote: Would it help if you split the file into individual records before processing them? I think it would help immensely I think. I'd prefer to fight my way thru this in CF if at all possible and things are just complex enough that I may need to go the route of stax. Outside my comfort zone in terms of installing it on the server but I can likely figure it out. This morning I investigated the possibility of dropping into java and reading one line at a time, parsing out the text in that line... Its what I do in this file's CSV-format predecessor. But there are no CR's or LF's in the file so that is out. Thanks very much for that post and your thoughts. Incidentally if anyone is looking for an xml reader/editor that can almost instantly read these kinds of moster files, check out the free firstobject xml editor. http://www.firstobject.com/ -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342360 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Big XML files processing Really s-l-o-w. Solution?
Hi, Matt...
I've been tinkering with parsing the xml file and, though
I haven't developed a full solution, I've made some progress
that I thought I'd share.
I created a MySQL 5 database called, xmlTest.
I created a MySQL 5 database table called, xmlBlob.
Fields:
- rowID (int, not null, auto increment)
- xmlContent (blob)
- comment (varchar, 200)
I then ran this code:
cfset filePath = #expandPath('data.xml')#
cfoutputfilePath = #filePath#/cfoutput (checking accuracy)
cfquery name=processXML datasource=xmlTest
select xmlContent
fromxmlBlob
limit 1
into@xml
/cfquery
Data extraction method 1:
cfquery name=extractData01 datasource=xmlTest
select extractValue(@xml, '//dataRecord[1]/recordID[1]')
recordID
/cfquery
pcfdump toString(extractData01.recordID) = cfdump
var=#toString(extractData01.recordID)#/p
Data extraction method 2:
cfquery name=extractData02 datasource=xmlTest
select extractValue(b.xmlContent, '//dataRecord/recordID')
recordID,
extractValue(b.xmlContent,
'//dataRecord/year') year,
extractValue(b.xmlContent,
'//dataRecord/make') make,
extractValue(b.xmlContent,
'//dataRecord/model') model
fromxmlBlob b
/cfquery
cfoutputextractData02.recordCount =
#extractData02.recordCount#/cfoutputbr
br
cfoutput query=extractData02
#toString(extractData02.recordID)#br
#toString(extractData02.year)#br
#toString(extractData02.make)#br
#toString(extractData02.model)#br
br
/cfoutput
---
When the code above is run in a browser, it outputs:
cfdump toString(extractData01.recordID) = 123
extractData02.recordCount = 1
123 124
1980 1981
Trabant Ford
TurboXL TurboDL
---
I was just to the point of figuring out how to get
the two records (I modified your data.xml file example so there
would be two records) to show separately, instead of
the data being in a list for the fields, when I saw your
post and thought I'd go ahead and respond so you could see
if this might be an approach that would work for you.
In the data extraction method 1 above, the idea is to insert
the xml file (which might be too large for this?)
into a blob field in a MySQL table, then read
the blob field and put the xml data file into a variable, '@xml',
then run a cfquery against the variable.
Data extraction method 2 skips reading the xml into a variable
and queries the blob field directly. This method uses the
MySQL 'xpath_expression' syntax to access data in multi-level
xml data files. With xpath, you can go as deeply into the levels
as needed.
Like I said, this is just the result of tinkering around with
what I've been finding and attempting to get some usable output.
Perhaps this will be of some help.
Rick
-Original Message-
From: Matt Robertson [mailto:websitema...@gmail.com]
Sent: Wednesday, February 16, 2011 10:18 PM
To: cf-talk
Subject: Re: Big XML files processing Really s-l-o-w. Solution?
On Wed, Feb 16, 2011 at 3:49 PM, Jochem van Dieten wrote:
Would it help if you split the file into individual records before
processing them?
I think it would help immensely I think. I'd prefer to fight my way
thru this in CF if at all possible and things are just complex enough
that I may need to go the route of stax. Outside my comfort zone in
terms of installing it on the server but I can likely figure it out.
This morning I investigated the possibility of dropping into java and
reading one line at a time, parsing out the text in that line... Its
what I do in this file's CSV-format predecessor. But there are no
CR's or LF's in the file so that is out.
Thanks very much for that post and your thoughts.
Incidentally if anyone is looking for an xml reader/editor that can
almost instantly read these kinds of moster files, check out the free
firstobject xml editor.
http://www.firstobject.com/
--
--m@Robertson--
Janitor, The Robertson Team
mysecretbase.com
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342361
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: JVM help!
NewGen can sometimes be a problem. The temptation with 64 bit is setting a
large initial heap size -Xms4096m. This can be quite fine however can lead to
problems in JVM performance since the JVM is not making good decisions how to
size the New Generation (which is made up of Eden and two survivor spaces).
What is the problem? Sometimes the NewGen starts out so large that a minor
Garbage Collection will try move large size objects in the Survivor Space and
the OldGen can not fit it right away.
Just to simplfy that a bit to draw a mental picture, say the OldGen is 4Gb in
size currently and 3.xGb used. A NewGen that is not well configured can have a
1Gb survivor space. The minor GC tries to tenure the 1Gb however it does not
fit in the 1Gb free space. Well what can happen some more of the heap could be
virtually committed or a major GC can run to try clear the OldGen. I am over
simplifying there however I hope you can see that can have an impact on your
performance.
The same situation can also apply on 32 bit where values like âXms1024m
âXmx1024m are set in JVM args that is a relatively large tenure of objects in
survivor space may not fit in to uncommitted OldGen space. Usually JVM will end
up making better decisions about the NewGen however till then there are some
performance hits.
So how to overcome that? Well JVM logging is one way another might be to use
JDK tools like jconsole or jvisualvm to see if JVM NewGen is making poor
decisions about sizing. Perhaps donât want to LOG or run JDK tools OK. Some
say use XX:NewRatio switches I like to use âXmn (some Java sources say not to
use Xmn). So that would make your JVM.CONFIG look this following. The Xmn value
may depend sometimes somewhere 96m to 348m or more would really need some
logging / JDK tools to be sure.
JVM.args=-server -Xms4096m -Xmx8192m -Dsun.io.useCanonCaches=false âXmn148m
-XX:PermSize=1024m -XX:MaxPermSize=1024m etc
HTH, Carl.
Hi everyone,
thanks for all the good feedback.
Let me first tell you WHY I think its a JVM issue, then show you what
results I have had.
We took the exact same code, and connected it to the exact same DB
(and DB
server) and ran the exact same code in isolation. The comparison was
outstanding about 2s on the old machine and 10s as you have seen on
the new.
The connection between the SQL and both web machines pinged at 1ms
We tried the connection from various locations and ruled out network
issues
(although there could be ). We looked at execution times and logged
the
information pretty heavily in the intensive parts of the app and saw
that,
on comparison between machines the differences. Given that, and the
isolation tests, the only thing we could see that was different was
the JVM
settings, thus the conclusion we came too.
so having taken some of the suggestions into account I have ended up
with
this setting:
java.args=-server -Xms4096m -Xmx8192m -Dsun.io.useCanonCaches=false
-XX:PermSize=1024m -XX:MaxPermSize=1024m
-Dsun.rmi.dgc.client.gcInterval=15
-Dsun.rmi.dgc.server.gcInterval=15 -XX:+UseParallelGC -Xbatch
-Dcoldfusion.rootDir={application.home}/
-Djava.security.policy={application.
home}/ servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/coldfusion.
policy
-Djava.security.auth.policy={application.
home }/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/neo_jaas.
policy
This puts us at an execution time parallel to the old 32 bit machine,
down
at 1.5-2 seconds. while this is a vast improvement I would hope that
more
gains could be had out of tuning the JVM some more. I will tweak the
GCinterval and the permSize tomorrow night and see if there is an
incremental gain to be had.
Any other suggestions would be great!
Thanks.
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342362
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: JVM help!
Where else to make gains in JVM? You say the server is Quad Core CPU. Suggest allow the Garbage Collector in your case -XX:+UseParallelGC to multi thread and use all those cores. The JVM is going to GC so reduce the time of the GCâs is desirable especially for the Full GC as part of the cycle is a Java stop all processing the shorter the duration the sooner Java / CF can get on with other things. JVM args would then look this: java.args=-server ...-XX:+UseParallelGC -XX:ParallelGCThreads=4 âXbatch etc HTH, Carl. Hi everyone, thanks for all the good feedback. Let me first tell you WHY I think its a JVM issue, then show you what results I have had. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342363 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: JVM help!
Some Java / CF systems I have seen have done well to change the collector in use. Currently you are running -XX:+UseParallelGC which is a throughput GC. Perhaps you can experiment with changing to a low pause GC? Caveat UseParallelGC may be working well for the environment and changing collector might be a retrograde step and could be hard to tell without some JVM logging. There are other GCâs -XX:+UseParallelOldGC is said to be good for large heaps. For incremental GC (low pause) JVM would look like: java.args=-server -Xms4096m -Xmx8192m -Dsun.io.useCanonCaches=false -XX:PermSize=1024m -XX:MaxPermSize=1024m -Dsun.rmi.dgc.client.gcInterval=15 -Dsun.rmi.dgc.server.gcInterval=15 -XX:+ UseParNewGC -Xincgc âXbatch etc HTH again, Carl. Hi everyone, thanks for all the good feedback. Let me first tell you WHY I think its a JVM issue, then show you what results I have had. We took the exact same code, and connected it to the exact same DB (and DB server) and ran the exact same code in isolation. The comparison was outstanding about 2s on the old machine and 10s as you have seen on the new. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342364 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
cf-talk replies
Hi Folks I have noticed recently that when I reply to a cf-talk email My email is not showing up in my email box. For example I recently replied to: Pete Ruckelhaus' email Here are my mail headers: From:Rob Voyle robvo...@voyle.com To: Pete Ruckelshaus pruckelsh...@gmail.com, cf-talk@houseoffusion.com Subject:Re: Using Google Apps/Gmail for CFMail - limitations? Date sent: Wed, 16 Feb 2011 18:23:01 -0800 I know Pete got my response because we have corresponded but I haven't seen my original response in the cf-talk. I have had this happen on several of my reply emails to the group. Did the group get my reply and is it only not being sent to me? Rob ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342365 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: cf-talk replies
the last (before this one) post from you i see in my mailbox is from Jan 26 to thread session Variables Security if you have posted in-between Jan 26 and today then your posts did not show up in the list... Azadi On 17/02/2011 14:42 , Rob Voyle wrote: Hi Folks I have noticed recently that when I reply to a cf-talk email My email is not showing up in my email box. For example I recently replied to: Pete Ruckelhaus' email Here are my mail headers: From:Rob Voylerobvo...@voyle.com To: Pete Ruckelshauspruckelsh...@gmail.com, cf-talk@houseoffusion.com Subject:Re: Using Google Apps/Gmail for CFMail - limitations? Date sent:Wed, 16 Feb 2011 18:23:01 -0800 I know Pete got my response because we have corresponded but I haven't seen my original response in the cf-talk. I have had this happen on several of my reply emails to the group. Did the group get my reply and is it only not being sent to me? Rob ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:342366 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
