Re: Resolved: Server Configuration Recommendations
Bravo Craig. Thanks for posting the solution to the 'net, as many people are running SQL 2k5 with Windows Server 2k3. I hope this will help someone else out as well. Thanks, Gary Opela, Jr Sr. Remedy Developer Leader Communications, Inc. 405 736 3211 -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Phil Murnane Sent: Tuesday, January 15, 2008 5:00 PM To: arslist@ARSLIST.ORG Subject: Re: Resolved: Server Configuration Recommendations Thanks, Craig! - Original Message From: Craig Carter [EMAIL PROTECTED] To: arslist@ARSLIST.ORG Sent: Tuesday, January 15, 2008 3:26:19 PM Subject: Resolved: Server Configuration Recommendations I wanted to follow-up and provide some changes I made that had a HUGE impact on performance and resolved the majority of our problem. Although we had 16GB on the system, the majority of it was not being used and performance monitor showed everything was waiting for disk. Average disk queue length was long and the bar never dropped off 100. The SQL Server service had allocated memory/virtual memory up to 1.7 GB but was holding steady at that amount. We're running Windows 2003 Enterprise (32-bit) and the documentation stated SQL Server 2005 will not exceed the virtual memory setting on Windows Server 2003 (32-bit) unless you enable AWE memory allocation. There was also an enable lock pages in memory option that I enabled for the account SQL Server was running under--but it appears this may not be needed unless you are running under Windows 2000 or Windows XP. The setting by default is off in SQL Server 2005. I also updated the Virtual memory settings on the server to system managed which increased it by 800%. After restarting SQL Server, the SQL Server service memory usage dropped from 1.7GB to 130M and the paging file jumped from about 2.6 GB to almost 7 GB. The processors are two to three times busier now but still only averaging about 30%. The physical memory in use jumped by 4 GB and you can tell SQL Server is now going well beyond the virtual server limits and 4GB limits imposed by the operating system. In summary, the database was not being allowed to exceed the virtual memory limits and 4GB operating system limit shared with everything else. Now that is has plenty of breathing room, it's flying right along. There is still a lot of tweaking to do but enabling AWE allocation made a huge difference. Thanks for all your suggestions. If you are running SQL Server 2005 on a Windows Server 32-bit version with more than 4GB, check this option out. Craig Carter ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Resolved: Server Configuration Recommendations
There is a lot more performance tuning to do but it was amazing the difference this made. I was curious why SQL Server wasn't taking advantage of the extra memory on the system. All of this is explained in the SQL Server 2005 online help. Craig Carter -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Opela, Gary L Contr OC-ALC/ITMA Sent: Wednesday, January 16, 2008 6:20 AM To: arslist@ARSLIST.ORG Subject: Re: Resolved: Server Configuration Recommendations Bravo Craig. Thanks for posting the solution to the 'net, as many people are running SQL 2k5 with Windows Server 2k3. I hope this will help someone else out as well. Thanks, Gary Opela, Jr Sr. Remedy Developer Leader Communications, Inc. 405 736 3211 -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Phil Murnane Sent: Tuesday, January 15, 2008 5:00 PM To: arslist@ARSLIST.ORG Subject: Re: Resolved: Server Configuration Recommendations Thanks, Craig! - Original Message From: Craig Carter [EMAIL PROTECTED] To: arslist@ARSLIST.ORG Sent: Tuesday, January 15, 2008 3:26:19 PM Subject: Resolved: Server Configuration Recommendations I wanted to follow-up and provide some changes I made that had a HUGE impact on performance and resolved the majority of our problem. Although we had 16GB on the system, the majority of it was not being used and performance monitor showed everything was waiting for disk. Average disk queue length was long and the bar never dropped off 100. The SQL Server service had allocated memory/virtual memory up to 1.7 GB but was holding steady at that amount. We're running Windows 2003 Enterprise (32-bit) and the documentation stated SQL Server 2005 will not exceed the virtual memory setting on Windows Server 2003 (32-bit) unless you enable AWE memory allocation. There was also an enable lock pages in memory option that I enabled for the account SQL Server was running under--but it appears this may not be needed unless you are running under Windows 2000 or Windows XP. The setting by default is off in SQL Server 2005. I also updated the Virtual memory settings on the server to system managed which increased it by 800%. After restarting SQL Server, the SQL Server service memory usage dropped from 1.7GB to 130M and the paging file jumped from about 2.6 GB to almost 7 GB. The processors are two to three times busier now but still only averaging about 30%. The physical memory in use jumped by 4 GB and you can tell SQL Server is now going well beyond the virtual server limits and 4GB limits imposed by the operating system. In summary, the database was not being allowed to exceed the virtual memory limits and 4GB operating system limit shared with everything else. Now that is has plenty of breathing room, it's flying right along. There is still a lot of tweaking to do but enabling AWE allocation made a huge difference. Thanks for all your suggestions. If you are running SQL Server 2005 on a Windows Server 32-bit version with more than 4GB, check this option out. Craig Carter ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Q: Server Configuration Recommendations
Perhaps but page 19 of the performance tuning whitepaper recommends 3 times the number of processors for Fast and 5 times the processors for List (24 and 40). We could also have as many as 400 users and also some web users so this is intended to cover future growth. Craig Carter -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Jarl Grøneng Sent: Tuesday, January 15, 2008 10:29 PM To: arslist@ARSLIST.ORG Subject: Re: Q: Server Configuration Recommendations Is not 24 fast and 40 list an overkill with 80-100 users? -- Jarl On Jan 15, 2008 3:41 PM, Craig Carter [EMAIL PROTECTED] wrote: ** All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Q: Server Configuration Recommendations
Running 300-350 concurrent users with a max of 10 fast and 12 list. (Asset mgmt, change mgmt and home grown HD.) -- Jarl On Jan 16, 2008 2:29 PM, Craig Carter [EMAIL PROTECTED] wrote: Perhaps but page 19 of the performance tuning whitepaper recommends 3 times the number of processors for Fast and 5 times the processors for List (24 and 40). We could also have as many as 400 users and also some web users so this is intended to cover future growth. Craig Carter -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Jarl Grøneng Sent: Tuesday, January 15, 2008 10:29 PM To: arslist@ARSLIST.ORG Subject: Re: Q: Server Configuration Recommendations Is not 24 fast and 40 list an overkill with 80-100 users? -- Jarl On Jan 15, 2008 3:41 PM, Craig Carter [EMAIL PROTECTED] wrote: ** All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Server Configuration Recommendations
We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ * The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank you. * ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Q: Server Configuration Recommendations
All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Server Configuration Recommendations
I personally tend to disagree with the notion of putting the DB on its own server. I prefer to put ARS and the DB on the same server and Midtier on its own to cut down on the database accesses and updates from having to be accomplished over the network. We have over 2,000 concurrent users, and our performance is outstanding. Considering your performance is slowing with a modest increase in the number of users, I would suspect a memory leak of some sort. Have you looked at RAM consumption in task manager? -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 9:13 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations ** Thanks Chad-that is my impression as well. There should be no reason we should be having these performance problems based on the server we just set up. We generate a lot of emails each day and have over 500K tickets but something else is obviously wrong. Craig Carter From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Hall Chad - chahal Sent: Tuesday, January 15, 2008 7:58 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Server Configuration Recommendations
Thanks Chad-that is my impression as well. There should be no reason we should be having these performance problems based on the server we just set up. We generate a lot of emails each day and have over 500K tickets but something else is obviously wrong. Craig Carter From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Hall Chad - chahal Sent: Tuesday, January 15, 2008 7:58 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Server Configuration Recommendations
Yes, I've monitored Task Manager extensively and I don't see any memory issues--in fact, we have almost 13GB available. ARSystem grows and decreases as needed but it's not anything significant or continually increasing. I would prefer to leave ARS and the DB on the same server as well. Our midtier server in on another server. Craig Carter -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Kaiser Norm E CIV USAF 96 CS/SCCE Sent: Tuesday, January 15, 2008 8:23 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations I personally tend to disagree with the notion of putting the DB on its own server. I prefer to put ARS and the DB on the same server and Midtier on its own to cut down on the database accesses and updates from having to be accomplished over the network. We have over 2,000 concurrent users, and our performance is outstanding. Considering your performance is slowing with a modest increase in the number of users, I would suspect a memory leak of some sort. Have you looked at RAM consumption in task manager? -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 9:13 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations ** Thanks Chad-that is my impression as well. There should be no reason we should be having these performance problems based on the server we just set up. We generate a lot of emails each day and have over 500K tickets but something else is obviously wrong. Craig Carter ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Server Configuration Recommendations
On Jan 15, 2008 10:22 AM, Kaiser Norm E CIV USAF 96 CS/SCCE [EMAIL PROTECTED] wrote: I personally tend to disagree with the notion of putting the DB on its own server. I prefer to put ARS and the DB on the same server and Midtier on its own to cut down on the database accesses and updates from having to be accomplished over the network. Local sockets will always be faster than IP sockets. We have over 2,000 concurrent users, and our performance is outstanding. Considering your performance is slowing with a modest increase in the number of users, I would suspect a memory leak of some sort. Have you looked at RAM consumption in task manager? -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 9:13 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations ** Thanks Chad-that is my impression as well. There should be no reason we should be having these performance problems based on the server we just set up. We generate a lot of emails each day and have over 500K tickets but something else is obviously wrong. Craig Carter From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Hall Chad - chahal Sent: Tuesday, January 15, 2008 7:58 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist
Re: Server Configuration Recommendations
Hi Craig, I would suggest looking at your applications and how they are designed before throwing more money at your hardware. As suggested, the API+SQL+FLTR+ESCL-logs will show where your server is spending it's energy. It may be a few really bad select-statements that fail to utilize your indexes, but it can just as well be a lot of semi-bad-statements. It is often easy to fix, but may sometimes require significant redesign of your applications. You will find a few key things to look for in the performance tuning presentation I did at UKRUG in 2006: http://rrr.se/doc/rrrukrug2006.pdf Best Regards - Misi, RRR AB, http://www.rrr.se Products from RRR Scandinavia: * RRR|License - Not enough Remedy licenses? Save money by optimizing. * RRR|Log - Performance issues or elusive bugs? Analyze your Remedy logs. * RRR|Translator - Manage and automate your language translations. Find these products, and many free tools and utilities, at http://rrr.se. We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ * The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank you. * ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor
Re: Server Configuration Recommendations
Very good point Axton. I've just never had the luxury of a server big enough to scale our system. It is too expensive. With that limitation, and a good gigabit network already in place, separate servers was the best choice for us. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Axton Sent: Tuesday, January 15, 2008 9:37 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations ** On Jan 15, 2008 10:22 AM, Kaiser Norm E CIV USAF 96 CS/SCCE [EMAIL PROTECTED] wrote: I personally tend to disagree with the notion of putting the DB on its own server. I prefer to put ARS and the DB on the same server and Midtier on its own to cut down on the database accesses and updates from having to be accomplished over the network. Local sockets will always be faster than IP sockets. We have over 2,000 concurrent users, and our performance is outstanding. Considering your performance is slowing with a modest increase in the number of users, I would suspect a memory leak of some sort. Have you looked at RAM consumption in task manager? -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 9:13 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations ** Thanks Chad-that is my impression as well. There should be no reason we should be having these performance problems based on the server we just set up. We generate a lot of emails each day and have over 500K tickets but something else is obviously wrong. Craig Carter From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Hall Chad - chahal Sent: Tuesday, January 15, 2008 7:58 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps
Re: Server Configuration Recommendations
Craig: What does the server resource utilization look like when you go from 80 to 100 users? Is there lots of paging or some heavy disk I/O? I'm sure you've looked in this direction, but perhaps we can offer a fresh pair of eyes. One thing I've seen kill database performance is if the DB is stored on a RAID 5 volume and there's lots of write activity. If the database starts looking like the bottleneck, this might be something to pursue. FWIW, --Phil - Original Message From: Axton [EMAIL PROTECTED] To: arslist@ARSLIST.ORG Sent: Tuesday, January 15, 2008 8:36:33 AM Subject: Re: Server Configuration Recommendations ** On Jan 15, 2008 10:22 AM, Kaiser Norm E CIV USAF 96 CS/SCCE [EMAIL PROTECTED] wrote: I personally tend to disagree with the notion of putting the DB on its own server. I prefer to put ARS and the DB on the same server and Midtier on its own to cut down on the database accesses and updates from having to be accomplished over the network. Local sockets will always be faster than IP sockets. We have over 2,000 concurrent users, and our performance is outstanding. Considering your performance is slowing with a modest increase in the number of users, I would suspect a memory leak of some sort. Have you looked at RAM consumption in task manager? -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 9:13 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations ** Thanks Chad-that is my impression as well. There should be no reason we should be having these performance problems based on the server we just set up. We generate a lot of emails each day and have over 500K tickets but something else is obviously wrong. Craig Carter From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Hall Chad - chahal Sent: Tuesday, January 15, 2008 7:58 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users
Re: Server Configuration Recommendations
Thanks Misi. I've already read the whitepaper and verified all the normal stuff and our network guys verified the connection should be 1GB so I'm leaning towards bad code at this point. I'll check out your guide and start digging through the logs. Craig Carter -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Misi Mladoniczky Sent: Tuesday, January 15, 2008 9:27 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations Hi Craig, I would suggest looking at your applications and how they are designed before throwing more money at your hardware. As suggested, the API+SQL+FLTR+ESCL-logs will show where your server is spending it's energy. It may be a few really bad select-statements that fail to utilize your indexes, but it can just as well be a lot of semi-bad-statements. It is often easy to fix, but may sometimes require significant redesign of your applications. You will find a few key things to look for in the performance tuning presentation I did at UKRUG in 2006: http://rrr.se/doc/rrrukrug2006.pdf Best Regards - Misi, RRR AB, http://www.rrr.se Products from RRR Scandinavia: * RRR|License - Not enough Remedy licenses? Save money by optimizing. * RRR|Log - Performance issues or elusive bugs? Analyze your Remedy logs. * RRR|Translator - Manage and automate your language translations. Find these products, and many free tools and utilities, at http://rrr.se. We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ * The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any
Re: Server Configuration Recommendations
Hi Phil, I believe it is RAID 5 and there is a lot of disk I/O but it doesn't seem to be excessive. What tools do you recommend to evaluate that other than Task Manager? It's a progressive thing-the more that log in, the worse it gets so I'm more inclined to believe we have some excessive database activity. I'm getting tired of getting beat up by the users so I'm looking for any recommendations of things to look for. Good to hear from you, Craig Carter From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Phil Murnane Sent: Tuesday, January 15, 2008 11:08 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations Craig: What does the server resource utilization look like when you go from 80 to 100 users? Is there lots of paging or some heavy disk I/O? I'm sure you've looked in this direction, but perhaps we can offer a fresh pair of eyes. One thing I've seen kill database performance is if the DB is stored on a RAID 5 volume and there's lots of write activity. If the database starts looking like the bottleneck, this might be something to pursue. FWIW, __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
RES: Server Configuration Recommendations
Hi, I have a question about this subject... How can I measure the memory consumption of my environment production??? It's through ARS logs? Database logs? Or exist a specify tool for this?? Thanks-4-All! Tadeu Augusto Dutra Pinto - IT Web Services ATM Cinq Technologies http://www.cinq.com.br [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Fone: 41 3018-2833 - Cinq Fone: 41 2107-5736 - HSBC Outsourcing - Confiabilidade, Inovação e Qualidade em T.I. De: Action Request System discussion list(ARSList) em nome de Craig Carter Enviada: ter 15/1/2008 16:18 Para: arslist@ARSLIST.ORG Assunto: Re: Server Configuration Recommendations Thanks Misi. I've already read the whitepaper and verified all the normal stuff and our network guys verified the connection should be 1GB so I'm leaning towards bad code at this point. I'll check out your guide and start digging through the logs. Craig Carter -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Misi Mladoniczky Sent: Tuesday, January 15, 2008 9:27 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations Hi Craig, I would suggest looking at your applications and how they are designed before throwing more money at your hardware. As suggested, the API+SQL+FLTR+ESCL-logs will show where your server is spending it's energy. It may be a few really bad select-statements that fail to utilize your indexes, but it can just as well be a lot of semi-bad-statements. It is often easy to fix, but may sometimes require significant redesign of your applications. You will find a few key things to look for in the performance tuning presentation I did at UKRUG in 2006: http://rrr.se/doc/rrrukrug2006.pdf Best Regards - Misi, RRR AB, http://www.rrr.se Products from RRR Scandinavia: * RRR|License - Not enough Remedy licenses? Save money by optimizing. * RRR|Log - Performance issues or elusive bugs? Analyze your Remedy logs. * RRR|Translator - Manage and automate your language translations. Find these products, and many free tools and utilities, at http://rrr.se. We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck
Re: Server Configuration Recommendations
SQL Server will have some tools to help you find those bottlenecks. 2005 is supposed to have a lot of new features, but I haven't used it. You can also use Performance Monitor and look at disk activity. I believe the counters come with an explanation of each metric and what acceptable values are. If not, you can Google them. We had ours on a RAID 5 on a SAN and we moved it to a larger RAID 10 set on a different SAN and it made a world of difference. We were seeing long waits in API and SQL calls, low CPU usage, and no network saturation. It wouldn't take long for a chain of blocked database processes to block all of our AR Server threads, resulting in slow response. Examination of the blocking processes showed they were all waiting on some process that held a lock on a table or index or whatever that all the other processes needed to access. Examination of our SAN showed it was being overworked. Switching to RAID 10 alleviated all those problems. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 12:27 PM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations Hi Phil, I believe it is RAID 5 and there is a lot of disk I/O but it doesn't seem to be excessive. What tools do you recommend to evaluate that other than Task Manager? It's a progressive thing-the more that log in, the worse it gets so I'm more inclined to believe we have some excessive database activity. I'm getting tired of getting beat up by the users so I'm looking for any recommendations of things to look for. Good to hear from you, Craig Carter From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Phil Murnane Sent: Tuesday, January 15, 2008 11:08 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations Craig: What does the server resource utilization look like when you go from 80 to 100 users? Is there lots of paging or some heavy disk I/O? I'm sure you've looked in this direction, but perhaps we can offer a fresh pair of eyes. One thing I've seen kill database performance is if the DB is stored on a RAID 5 volume and there's lots of write activity. If the database starts looking like the bottleneck, this might be something to pursue. FWIW, __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ * The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank you. * ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Server Configuration Recommendations
Craig: Here's a nifty little tutorial on using the Windows Server 2003 performance monitor to set up a daily log to log system performance and utilization: http://www.computerperformance.co.uk/HealthCheck/FirstLog.htm. It might help you pinpoint what's happening when. Just another Mr. Obvious thought - Do you have antivirus software on your server? Is it possible it's scheduled for a time somewhere around lunchtime? A long time ago in a galaxy far, far away, I set up a Remedy server for a site that had this problem. They ran antivirus from a parent server, and the parent server dictated that scans be run at noon (when users are at lunch). Of course, I didn't work fulltime at this site so I wasn't aware of their antivirus configuration. The scan would kick off and then take several hours to complete, gobbling up processor and disk access. Someone finally complained about the slowdown and I requested that the server be added to an exclusion such that antivirus would be kicked of at 2AM rather than noon. -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 12:27 PM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations ** Hi Phil, I believe it is RAID 5 and there is a lot of disk I/O but it doesn't seem to be excessive. What tools do you recommend to evaluate that other than Task Manager? It's a progressive thing-the more that log in, the worse it gets so I'm more inclined to believe we have some excessive database activity. I'm getting tired of getting beat up by the users so I'm looking for any recommendations of things to look for. Good to hear from you, Craig Carter From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Phil Murnane Sent: Tuesday, January 15, 2008 11:08 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations Craig: What does the server resource utilization look like when you go from 80 to 100 users? Is there lots of paging or some heavy disk I/O? I'm sure you've looked in this direction, but perhaps we can offer a fresh pair of eyes. One thing I've seen kill database performance is if the DB is stored on a RAID 5 volume and there's lots of write activity. If the database starts looking like the bottleneck, this might be something to pursue. FWIW, __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
FW: Server Configuration Recommendations
There are tools for this, but you can use task manager (in Windows) to do this: Here's an excerpt from an article: Tip One - OK...I want to pump some life into my old computer. How much RAM should I buy? In a recent installment of TOTW, I imparted a few ideas on how to breathe new life into aging computers. My number one recommendation was to buy more RAM. I also wrote that to figure out what kind of RAM you needed for your particular computer, you should pull one of the RAM sticks off the motherboard and read the specs off of it. But what I left open was the matter of how much RAM to buy. In that installment of TOTW, I wrote that you should probably just double up on whatever amount you currently have. In this installment, I share a better, more precise method of determining just how much memory you need. Here's what you do: * First, turn your computer on and leave it on for two or three days. Use it during that time the way you normally would--surf the Net, read e-mail, play your games, do your work (EPRs, PowerPoint presentations), etc. If you do any type of graphics work, like manipulating pictures from a digital camera, be sure to do some of that, too. And if you commonly open multiple programs at once (like I do) be sure to do that, too. * After the two or three day observation period, click CTL+ALT+DEL and click the TASK MANAGER button. That opens the Task Manager application. Now click on the PERFORMANCE tab, which looks like this: * Now notice the items I have circled in the image above. The item circled in blue is the total amount of RAM I have installed in my computer. The item circled in red is the highest amount of RAM my computer has needed to function properly since I first turned it on this morning. Notice that the number circled in red is higher than the number circled in blue. That's a bad thing. Now you might be wondering, How did Norm's computer use more RAM than it actually has? The answer lies in something called virtual memory. Virtual memory is simulated RAM. It's a trick your computer uses to fool itself into thinking it has more RAM than it actually does. How it does this is by using some of your hard drive as RAM. Sounds great, and it's a clever trick, but the problem is, compared to real RAM, your hard drive is slow. I mean, sloow. We're talking F-22 vs. bicycle here. So every time my computer needs to use more RAM than it actually has, it kicks in the virtual memory trick. The problem is, my computer slows way down when it does that because using the hard drive as RAM is a slow process. * Compare the two numbers on your computer. Do the same thing I did. To read these number in megabytes (MB) instead of kilobytes (KB), just divide the numbers by 1000. So my total memory is 259MB and my peak usage was 343MB. * If your PEAK number is higher than your TOTAL number, go buy more RAM. Buy at least the amount shown as PEAK. When you do buy RAM, I recommend that you buy the largest stick you can afford, and buy one stick instead of two. That way you'll keep an empty slot available for any future upgrade. ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Server Configuration Recommendations
You didn't specify what server OS... For Windows you can use Task Manager and Performance Monitor For Unix there are tools like top and prstat -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Tadeu Augusto Dutra Pinto Sent: Tuesday, January 15, 2008 12:30 PM To: arslist@ARSLIST.ORG Subject: RES: Server Configuration Recommendations Hi, I have a question about this subject... How can I measure the memory consumption of my environment production??? It's through ARS logs? Database logs? Or exist a specify tool for this?? Thanks-4-All! Tadeu Augusto Dutra Pinto - IT Web Services ATM Cinq Technologies http://www.cinq.com.br [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Fone: 41 3018-2833 - Cinq Fone: 41 2107-5736 - HSBC Outsourcing - Confiabilidade, Inovação e Qualidade em T.I. De: Action Request System discussion list(ARSList) em nome de Craig Carter Enviada: ter 15/1/2008 16:18 Para: arslist@ARSLIST.ORG Assunto: Re: Server Configuration Recommendations Thanks Misi. I've already read the whitepaper and verified all the normal stuff and our network guys verified the connection should be 1GB so I'm leaning towards bad code at this point. I'll check out your guide and start digging through the logs. Craig Carter -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Misi Mladoniczky Sent: Tuesday, January 15, 2008 9:27 AM To: arslist@ARSLIST.ORG Subject: Re: Server Configuration Recommendations Hi Craig, I would suggest looking at your applications and how they are designed before throwing more money at your hardware. As suggested, the API+SQL+FLTR+ESCL-logs will show where your server is spending it's energy. It may be a few really bad select-statements that fail to utilize your indexes, but it can just as well be a lot of semi-bad-statements. It is often easy to fix, but may sometimes require significant redesign of your applications. You will find a few key things to look for in the performance tuning presentation I did at UKRUG in 2006: http://rrr.se/doc/rrrukrug2006.pdf Best Regards - Misi, RRR AB, http://www.rrr.se Products from RRR Scandinavia: * RRR|License - Not enough Remedy licenses? Save money by optimizing. * RRR|Log - Performance issues or elusive bugs? Analyze your Remedy logs. * RRR|Translator - Manage and automate your language translations. Find these products, and many free tools and utilities, at http://rrr.se. We've ran over 600 concurrent users on a 2x1.4 GHz with 1 GB of memory without any issues. And that's a system with millions of tickets and over 50,000 emails sent each day on only 20 fast, 20 list, plus some private threads. We have two of these so one doesn't normally carry that load, but it has at times. So I don't think it's the size of your server. Ordinarily I would suggest you split the database out onto its own server. But given how big your current hardware is, and how small your current user base is, I see no need for that just yet. You seem to have plenty of hardware for that size of a system. I would use something like BMC Log Analyzer to analyze your API and SQL logs from a slowdown. That will tell you if you've got long running SQL or API calls, if you have little or no idle time on some threads (queuing), etc. It's available on the BMCDN last time I checked. I believe Misi has a log analysis tool as well at www.rrr.se http://www.rrr.se/ , but I haven't used it myself. If you see that your API calls are big, and your SQL calls are big, you've got a problem with your database. In that case, look at database metrics or hardware metrics to see if you have disk contention problems. Once we tuned our AR Server almost all of our bottlenecks were in the database, and mainly with disk contention. Chad Hall (501) 342-2650 From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Craig Carter Sent: Tuesday, January 15, 2008 8:41 AM To: arslist@ARSLIST.ORG Subject: Q: Server Configuration Recommendations All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps-no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish
Re: FW: Server Configuration Recommendations
Funny thing about Windows, why does it even use the page file when physical memory is available. Solaris and BSD do not do this. The only time the page file is used on these OS's is when physical memory is exhausted. Looking at my desktop right now: Total Physical: 2086928k Physical Available: 730544k Page File: 1189008k Axton Grams On Jan 15, 2008 2:11 PM, Kaiser Norm E CIV USAF 96 CS/SCCE [EMAIL PROTECTED] wrote: There are tools for this, but you can use task manager (in Windows) to do this: Here's an excerpt from an article: Tip One - OK...I want to pump some life into my old computer. How much RAM should I buy? In a recent installment of TOTW, I imparted a few ideas on how to breathe new life into aging computers. My number one recommendation was to buy more RAM. I also wrote that to figure out what kind of RAM you needed for your particular computer, you should pull one of the RAM sticks off the motherboard and read the specs off of it. But what I left open was the matter of how much RAM to buy. In that installment of TOTW, I wrote that you should probably just double up on whatever amount you currently have. In this installment, I share a better, more precise method of determining just how much memory you need. Here's what you do: * First, turn your computer on and leave it on for two or three days. Use it during that time the way you normally would--surf the Net, read e-mail, play your games, do your work (EPRs, PowerPoint presentations), etc. If you do any type of graphics work, like manipulating pictures from a digital camera, be sure to do some of that, too. And if you commonly open multiple programs at once (like I do) be sure to do that, too. * After the two or three day observation period, click CTL+ALT+DEL and click the TASK MANAGER button. That opens the Task Manager application. Now click on the PERFORMANCE tab, which looks like this: * Now notice the items I have circled in the image above. The item circled in blue is the total amount of RAM I have installed in my computer. The item circled in red is the highest amount of RAM my computer has needed to function properly since I first turned it on this morning. Notice that the number circled in red is higher than the number circled in blue. That's a bad thing. Now you might be wondering, How did Norm's computer use more RAM than it actually has? The answer lies in something called virtual memory. Virtual memory is simulated RAM. It's a trick your computer uses to fool itself into thinking it has more RAM than it actually does. How it does this is by using some of your hard drive as RAM. Sounds great, and it's a clever trick, but the problem is, compared to real RAM, your hard drive is slow. I mean, sloow. We're talking F-22 vs. bicycle here. So every time my computer needs to use more RAM than it actually has, it kicks in the virtual memory trick. The problem is, my computer slows way down when it does that because using the hard drive as RAM is a slow process. * Compare the two numbers on your computer. Do the same thing I did. To read these number in megabytes (MB) instead of kilobytes (KB), just divide the numbers by 1000. So my total memory is 259MB and my peak usage was 343MB. * If your PEAK number is higher than your TOTAL number, go buy more RAM. Buy at least the amount shown as PEAK. When you do buy RAM, I recommend that you buy the largest stick you can afford, and buy one stick instead of two. That way you'll keep an empty slot available for any future upgrade. ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: FW: Server Configuration Recommendations
Isn't the page file the system swap file? I think it displays it's size, but doesn't actually use it, until the available RAM has been all used up. Rick On 1/15/08, Axton [EMAIL PROTECTED] wrote: ** Funny thing about Windows, why does it even use the page file when physical memory is available. Solaris and BSD do not do this. The only time the page file is used on these OS's is when physical memory is exhausted. Looking at my desktop right now: Total Physical: 2086928k Physical Available: 730544k Page File: 1189008k Axton Grams On Jan 15, 2008 2:11 PM, Kaiser Norm E CIV USAF 96 CS/SCCE [EMAIL PROTECTED] wrote: There are tools for this, but you can use task manager (in Windows) to do this: Here's an excerpt from an article: Tip One - OK...I want to pump some life into my old computer. How much RAM should I buy? In a recent installment of TOTW, I imparted a few ideas on how to breathe new life into aging computers. My number one recommendation was to buy more RAM. I also wrote that to figure out what kind of RAM you needed for your particular computer, you should pull one of the RAM sticks off the motherboard and read the specs off of it. But what I left open was the matter of how much RAM to buy. In that installment of TOTW, I wrote that you should probably just double up on whatever amount you currently have. In this installment, I share a better, more precise method of determining just how much memory you need. Here's what you do: * First, turn your computer on and leave it on for two or three days. Use it during that time the way you normally would--surf the Net, read e-mail, play your games, do your work (EPRs, PowerPoint presentations), etc. If you do any type of graphics work, like manipulating pictures from a digital camera, be sure to do some of that, too. And if you commonly open multiple programs at once (like I do) be sure to do that, too. * After the two or three day observation period, click CTL+ALT+DEL and click the TASK MANAGER button. That opens the Task Manager application. Now click on the PERFORMANCE tab, which looks like this: * Now notice the items I have circled in the image above. The item circled in blue is the total amount of RAM I have installed in my computer. The item circled in red is the highest amount of RAM my computer has needed to function properly since I first turned it on this morning. Notice that the number circled in red is higher than the number circled in blue. That's a bad thing. Now you might be wondering, How did Norm's computer use more RAM than it actually has? The answer lies in something called virtual memory. Virtual memory is simulated RAM. It's a trick your computer uses to fool itself into thinking it has more RAM than it actually does. How it does this is by using some of your hard drive as RAM. Sounds great, and it's a clever trick, but the problem is, compared to real RAM, your hard drive is slow. I mean, sloow. We're talking F-22 vs. bicycle here. So every time my computer needs to use more RAM than it actually has, it kicks in the virtual memory trick. The problem is, my computer slows way down when it does that because using the hard drive as RAM is a slow process. * Compare the two numbers on your computer. Do the same thing I did. To read these number in megabytes (MB) instead of kilobytes (KB), just divide the numbers by 1000. So my total memory is 259MB and my peak usage was 343MB. * If your PEAK number is higher than your TOTAL number, go buy more RAM. Buy at least the amount shown as PEAK. When you do buy RAM, I recommend that you buy the largest stick you can afford, and buy one stick instead of two. That way you'll keep an empty slot available for any future upgrade. ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: FW: Server Configuration Recommendations
From my perspective, I would like to be able to pin an application into memory, and you cannot do that with Windows as far as I know. I believe that's something that may be available in the next windows server version Guillaume -Original Message- From: Action Request System discussion list(ARSList) on behalf of Axton Sent: Tue 01/15/08 3:12 PM To: arslist@ARSLIST.ORG Subject: Re: FW: Server Configuration Recommendations Funny thing about Windows, why does it even use the page file when physical memory is available. Solaris and BSD do not do this. The only time the page file is used on these OS's is when physical memory is exhausted. Looking at my desktop right now: Total Physical: 2086928k Physical Available: 730544k Page File: 1189008k Axton Grams On Jan 15, 2008 2:11 PM, Kaiser Norm E CIV USAF 96 CS/SCCE [EMAIL PROTECTED] wrote: There are tools for this, but you can use task manager (in Windows) to do this: Here's an excerpt from an article: Tip One - OK...I want to pump some life into my old computer. How much RAM should I buy? In a recent installment of TOTW, I imparted a few ideas on how to breathe new life into aging computers. My number one recommendation was to buy more RAM. I also wrote that to figure out what kind of RAM you needed for your particular computer, you should pull one of the RAM sticks off the motherboard and read the specs off of it. But what I left open was the matter of how much RAM to buy. In that installment of TOTW, I wrote that you should probably just double up on whatever amount you currently have. In this installment, I share a better, more precise method of determining just how much memory you need. Here's what you do: * First, turn your computer on and leave it on for two or three days. Use it during that time the way you normally would--surf the Net, read e-mail, play your games, do your work (EPRs, PowerPoint presentations), etc. If you do any type of graphics work, like manipulating pictures from a digital camera, be sure to do some of that, too. And if you commonly open multiple programs at once (like I do) be sure to do that, too. * After the two or three day observation period, click CTL+ALT+DEL and click the TASK MANAGER button. That opens the Task Manager application. Now click on the PERFORMANCE tab, which looks like this: * Now notice the items I have circled in the image above. The item circled in blue is the total amount of RAM I have installed in my computer. The item circled in red is the highest amount of RAM my computer has needed to function properly since I first turned it on this morning. Notice that the number circled in red is higher than the number circled in blue. That's a bad thing. Now you might be wondering, How did Norm's computer use more RAM than it actually has? The answer lies in something called virtual memory. Virtual memory is simulated RAM. It's a trick your computer uses to fool itself into thinking it has more RAM than it actually does. How it does this is by using some of your hard drive as RAM. Sounds great, and it's a clever trick, but the problem is, compared to real RAM, your hard drive is slow. I mean, sloow. We're talking F-22 vs. bicycle here. So every time my computer needs to use more RAM than it actually has, it kicks in the virtual memory trick. The problem is, my computer slows way down when it does that because using the hard drive as RAM is a slow process. * Compare the two numbers on your computer. Do the same thing I did. To read these number in megabytes (MB) instead of kilobytes (KB), just divide the numbers by 1000. So my total memory is 259MB and my peak usage was 343MB. * If your PEAK number is higher than your TOTAL number, go buy more RAM. Buy at least the amount shown as PEAK. When you do buy RAM, I recommend that you buy the largest stick you can afford, and buy one stick instead of two. That way you'll keep an empty slot available for any future upgrade. ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: FW: Server Configuration Recommendations
Windows tries to maximize the availability of physical RAM. When an application is inactive or minimized, Windows pushes the app's data to the page file to make physical RAM available. Example: Say I'm editing an image with Photoshop. That eats up a lot of memory. If I minimize it to work on a Word document, Windows put the Photoshop data in the page file to make more RAM available for my active application. -Original Message- From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Axton Sent: Tuesday, January 15, 2008 2:12 PM To: arslist@ARSLIST.ORG Subject: Re: FW: Server Configuration Recommendations ** Funny thing about Windows, why does it even use the page file when physical memory is available. Solaris and BSD do not do this. The only time the page file is used on these OS's is when physical memory is exhausted. Looking at my desktop right now: Total Physical: 2086928k Physical Available: 730544k Page File: 1189008k Axton Grams On Jan 15, 2008 2:11 PM, Kaiser Norm E CIV USAF 96 CS/SCCE [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: There are tools for this, but you can use task manager (in Windows) to do this: Here's an excerpt from an article: Tip One - OK...I want to pump some life into my old computer. How much RAM should I buy? In a recent installment of TOTW, I imparted a few ideas on how to breathe new life into aging computers. My number one recommendation was to buy more RAM. I also wrote that to figure out what kind of RAM you needed for your particular computer, you should pull one of the RAM sticks off the motherboard and read the specs off of it. But what I left open was the matter of how much RAM to buy. In that installment of TOTW, I wrote that you should probably just double up on whatever amount you currently have. In this installment, I share a better, more precise method of determining just how much memory you need. Here's what you do: * First, turn your computer on and leave it on for two or three days. Use it during that time the way you normally would--surf the Net, read e-mail, play your games, do your work (EPRs, PowerPoint presentations), etc. If you do any type of graphics work, like manipulating pictures from a digital camera, be sure to do some of that, too. And if you commonly open multiple programs at once (like I do) be sure to do that, too. * After the two or three day observation period, click CTL+ALT+DEL and click the TASK MANAGER button. That opens the Task Manager application. Now click on the PERFORMANCE tab, which looks like this: * Now notice the items I have circled in the image above. The item circled in blue is the total amount of RAM I have installed in my computer. The item circled in red is the highest amount of RAM my computer has needed to function properly since I first turned it on this morning. Notice that the number circled in red is higher than the number circled in blue. That's a bad thing. Now you might be wondering, How did Norm's computer use more RAM than it actually has? The answer lies in something called virtual memory. Virtual memory is simulated RAM. It's a trick your computer uses to fool itself into thinking it has more RAM than it actually does. How it does this is by using some of your hard drive as RAM. Sounds great, and it's a clever trick, but the problem is, compared to real RAM, your hard drive is slow. I mean, sloow. We're talking F-22 vs. bicycle here. So every time my computer needs to use more RAM than it actually has, it kicks in the virtual memory trick. The problem is, my computer slows way down when it does that because using the hard drive as RAM is a slow process. * Compare the two numbers on your computer. Do the same thing I did. To read these number in megabytes (MB) instead of kilobytes (KB), just divide the numbers by 1000. So my total memory is 259MB and my peak usage was 343MB. * If your PEAK number is higher than your TOTAL number, go buy more RAM. Buy at least the amount shown as PEAK. When you do buy RAM, I recommend that you buy the largest stick you can afford, and buy one stick instead of two. That way you'll keep an empty slot available for any future upgrade. ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor
Resolved: Server Configuration Recommendations
I wanted to follow-up and provide some changes I made that had a HUGE impact on performance and resolved the majority of our problem. Although we had 16GB on the system, the majority of it was not being used and performance monitor showed everything was waiting for disk. Average disk queue length was long and the bar never dropped off 100. The SQL Server service had allocated memory/virtual memory up to 1.7 GB but was holding steady at that amount. We're running Windows 2003 Enterprise (32-bit) and the documentation stated SQL Server 2005 will not exceed the virtual memory setting on Windows Server 2003 (32-bit) unless you enable AWE memory allocation. There was also an enable lock pages in memory option that I enabled for the account SQL Server was running under--but it appears this may not be needed unless you are running under Windows 2000 or Windows XP. The setting by default is off in SQL Server 2005. I also updated the Virtual memory settings on the server to system managed which increased it by 800%. After restarting SQL Server, the SQL Server service memory usage dropped from 1.7GB to 130M and the paging file jumped from about 2.6 GB to almost 7 GB. The processors are two to three times busier now but still only averaging about 30%. The physical memory in use jumped by 4 GB and you can tell SQL Server is now going well beyond the virtual server limits and 4GB limits imposed by the operating system. In summary, the database was not being allowed to exceed the virtual memory limits and 4GB operating system limit shared with everything else. Now that is has plenty of breathing room, it's flying right along. There is still a lot of tweaking to do but enabling AWE allocation made a huge difference. Thanks for all your suggestions. If you are running SQL Server 2005 on a Windows Server 32-bit version with more than 4GB, check this option out. Craig Carter ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Resolved: Server Configuration Recommendations
Thanks, Craig! - Original Message From: Craig Carter [EMAIL PROTECTED] To: arslist@ARSLIST.ORG Sent: Tuesday, January 15, 2008 3:26:19 PM Subject: Resolved: Server Configuration Recommendations I wanted to follow-up and provide some changes I made that had a HUGE impact on performance and resolved the majority of our problem. Although we had 16GB on the system, the majority of it was not being used and performance monitor showed everything was waiting for disk. Average disk queue length was long and the bar never dropped off 100. The SQL Server service had allocated memory/virtual memory up to 1.7 GB but was holding steady at that amount. We're running Windows 2003 Enterprise (32-bit) and the documentation stated SQL Server 2005 will not exceed the virtual memory setting on Windows Server 2003 (32-bit) unless you enable AWE memory allocation. There was also an enable lock pages in memory option that I enabled for the account SQL Server was running under--but it appears this may not be needed unless you are running under Windows 2000 or Windows XP. The setting by default is off in SQL Server 2005. I also updated the Virtual memory settings on the server to system managed which increased it by 800%. After restarting SQL Server, the SQL Server service memory usage dropped from 1.7GB to 130M and the paging file jumped from about 2.6 GB to almost 7 GB. The processors are two to three times busier now but still only averaging about 30%. The physical memory in use jumped by 4 GB and you can tell SQL Server is now going well beyond the virtual server limits and 4GB limits imposed by the operating system. In summary, the database was not being allowed to exceed the virtual memory limits and 4GB operating system limit shared with everything else. Now that is has plenty of breathing room, it's flying right along. There is still a lot of tweaking to do but enabling AWE allocation made a huge difference. Thanks for all your suggestions. If you are running SQL Server 2005 on a Windows Server 32-bit version with more than 4GB, check this option out. Craig Carter ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
Re: Q: Server Configuration Recommendations
Is not 24 fast and 40 list an overkill with 80-100 users? -- Jarl On Jan 15, 2008 3:41 PM, Craig Carter [EMAIL PROTECTED] wrote: ** All, We're running into some performance problems recently. I upgraded the server a few days ago (operating system, database, CPUs, memory, queues, etc, and it hasn't helped much. Basically, it takes longer to search and create tickets as more and more people log in (as expected) but the server has plenty of available CPU, plenty of memory, and plenty of bandwidth so it appears there is a bottleneck somewhere. 8 CPUs 16GB Memory Windows Server 2003 Enterprise SQL Server 2005 Enterprise ARS v7.0.1 P5 (CSS and custom apps—no ITSM) 24 Fast, 40 List It flies with about 40 people, becomes sluggish with 80, and gets real slow with 100. I would expect this system to be able to handle a much larger load. Since the running CPU usage and disk usage is fairly low, I'm looking for advice. Everything is currently installed on the same server and on the same drive (although these are raid drives). Is it possible we're seeing contention over disk resources and I/O? Any advice on determining where the bottleneck is or from people administering a large number of users? How much advantage would be gained by running the AR Server on another drive or box separate from the database? Is it reasonable to expect to only get 100 concurrent users (using the WUT) on a server of this size? Looking in the docs and whitepapers but any advice would be helpful since this is impacting us now. Craig Carter __Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are html___ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: Where the Answers Are
