Re: [U2] Extreme slow down on Universe periodically
Thanks to all that have replied. We're still busy researching and testing but we did catch a break yesterday and it appears that the problem could be the same that Wol mentioned. We have had reports of CPU spikes in the past, but these were only reported in off hours and usually the following day or later. Yesterday however, they noticed a process that was Spiking using 100% CPU. We were able to track it to the user and find out exactly what they were doing. The user confirmed that they would just exit from the telnet session window (Windows Telnet) and not log off properly. When I tested this, the Process immediately began spiking. We're thinking and hoping that the slow downs that we have experienced are the cumulative affects of multiple bad disconnects and CPU spikes. I have contacted Rocket to see if/when this issue has been resolved and we will keep monitoring the server. Again thanks for the help and input. We're still going through several of the suggestions to see what impact they may have. Curt Stewart -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, April 08, 2011 11:54 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Extreme slow down on Universe periodically On 08/04/11 14:04, Curt Stewart wrote: Has anyone in the group experienced this issue and found a resolution? Any and all suggestions would be appreciated. Environment Universe 10.2.10 Windows Server 2003 Virtual Machine (sorry don't know any details on this component) 34 user license, only about 15 active This is an old problem, that might well be resolved on 10.2, but do you have any rogue uvtelnets? We used to get that - they'd get disconnected somehow from their user, and then grab 100% cpu. Kill them, and things went back to normal (we had to crash them, they wouldn't kill, by attaching the debugger or something). Charles mentioned the network. Are your network buffers too large? I'm beginning to pick up from my reading on a hardware/software mismatch at the TCP level, and network slowdown (and catastrophic failure) are the result. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
This supports my contention that Windows' telnet client is not robust enough for use in a production environment. [Ad] Anzio, in this situation, will properly close down the host connection before quitting. In addition, it can be configured so that it is not possible (short of Task Manager or reboot) to quit from the client side. This then coerces the user into exiting from the menu system on the server, clear out to where the telnetd or sshd closes the connection. Then the client side (Anzio) reacts to this and quits. It avoids a lot of problems. On Tue, 12 Apr 2011, Curt Stewart wrote: Thanks to all that have replied. We're still busy researching and testing but we did catch a break yesterday and it appears that the problem could be the same that Wol mentioned. We have had reports of CPU spikes in the past, but these were only reported in off hours and usually the following day or later. Yesterday however, they noticed a process that was Spiking using 100% CPU. We were able to track it to the user and find out exactly what they were doing. The user confirmed that they would just exit from the telnet session window (Windows Telnet) and not log off properly. When I tested this, the Process immediately began spiking. We're thinking and hoping that the slow downs that we have experienced are the cumulative affects of multiple bad disconnects and CPU spikes. I have contacted Rocket to see if/when this issue has been resolved and we will keep monitoring the server. Again thanks for the help and input. We're still going through several of the suggestions to see what impact they may have. Curt Stewart -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, April 08, 2011 11:54 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Extreme slow down on Universe periodically On 08/04/11 14:04, Curt Stewart wrote: Has anyone in the group experienced this issue and found a resolution? Any and all suggestions would be appreciated. Environment Universe 10.2.10 Windows Server 2003 Virtual Machine (sorry don't know any details on this component) 34 user license, only about 15 active This is an old problem, that might well be resolved on 10.2, but do you have any rogue uvtelnets? We used to get that - they'd get disconnected somehow from their user, and then grab 100% cpu. Kill them, and things went back to normal (we had to crash them, they wouldn't kill, by attaching the debugger or something). Charles mentioned the network. Are your network buffers too large? I'm beginning to pick up from my reading on a hardware/software mismatch at the TCP level, and network slowdown (and catastrophic failure) are the result. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users Regards, Bob Rasmussen, President, Rasmussen Software, Inc. personal e-mail: r...@anzio.com company e-mail: r...@anzio.com voice: (US) 503-624-0360 (9:00-6:00 Pacific Time) fax: (US) 503-624-0760 web: http://www.anzio.com street address: Rasmussen Software, Inc. 10240 SW Nimbus, Suite L9 Portland, OR 97223 USA ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
On 12/04/11 19:37, Curt Stewart wrote: Thanks to all that have replied. We're still busy researching and testing but we did catch a break yesterday and it appears that the problem could be the same that Wol mentioned. We have had reports of CPU spikes in the past, but these were only reported in off hours and usually the following day or later. Yesterday however, they noticed a process that was Spiking using 100% CPU. We were able to track it to the user and find out exactly what they were doing. The user confirmed that they would just exit from the telnet session window (Windows Telnet) and not log off properly. When I tested this, the Process immediately began spiking. Okay, if that's what it looks like ... by the way, Bob, we used wIntegrate, and not Windows Telnet ... We're thinking and hoping that the slow downs that we have experienced are the cumulative affects of multiple bad disconnects and CPU spikes. I have contacted Rocket to see if/when this issue has been resolved and we will keep monitoring the server. If you get a report of a slowdown, get onto the server and run Task Manager (teaching grandma to suck eggs, but right click the task bar and select it). Sort processes by cpu and see if uvtelnet has floated to the top :-) Right click the process and kill it. As I said, though, we found that end process wouldn't do it. Selecting debug, however, did. Again thanks for the help and input. We're still going through several of the suggestions to see what impact they may have. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
Thanks to all that replied. I'll be following up on your suggestions and comments today, as well as several responses I received off-list. When we find an answer I'll post it for reference. I'm still hoping that it's something easy that I just haven't been able to pin-point yet. Thanks again. Curt Stewart -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bob Woodward Sent: Friday, April 08, 2011 5:55 PM To: U2 Users List Subject: Re: [U2] Extreme slow down on Universe periodically I'm sure you'll already have this in mind but I see the slow down as a symptom that can be caused by any number of things so even though I'm sure collaboration will be very helpful, it's not guaranteed that the same problem would exist in both locations. What may have passed inspection at your clients site may be the cause at Curt's client site. But you guys already knew that, I'm sure, so I'm just stating the obvious. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Friday, April 08, 2011 3:34 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Extreme slow down on Universe periodically From: Curt Stewart My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. Curt, one of our clients has a large site where this has been a chronic issue for about a year. They've done all the exercises with process monitors, network monitors, etc. Rocket has been all over this but I'm not sure they have a solid resolution. I'll link you guys up off-list so that you may be able to benefit from their experience. Best, T Tony Gravagno Nebula Research and Development TG@ remove.pleaseNebula-RnD.com remove.pleaseNebula-RnD.com/blog Visit PickWiki.com! Contribute! http://Twitter.com/TonyGravagno ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] Extreme slow down on Universe periodically
Has anyone in the group experienced this issue and found a resolution? My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. This affects both existing processes (menu selections, data entry, reports) and logins. Their system admins (IBM) reports that it is a Universe or application problem. I havent been able to identify any runaway processes or hogs. The log files that Ive looked at havent shed any light on the matter. Could there be a log file that Im overlooking that might need to be cleared? But if its an undersized file, why wouldnt it be a consistent problem? Im hoping that the group might be able to suggest some things to look at and consider. Any and all suggestions would be appreciated. Environment Universe 10.2.10 Windows Server 2003 Virtual Machine (sorry dont know any details on this component) 34 user license, only about 15 active Thanks for your help. Curt Stewart TRI-SYS Consulting ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically [not-secure]
1) First suspicion is the virtual-ness is the UniVerse server getting whacked by the virtual server hyperviser thingy? (Thus exhausting my knowledge of VM technical speak...) 2) RAM, RAM, RAM, DISK, DISK, DISK. Make sure your machine has lots of both... Stephen O'Neal, then of IBM (hopefully now of Rocket) put it best when he said you want your machine to be a disk WRITING machine. 3) Get off of Windows? Could the slow down be matching Anti-Virus/Malware scans? 4) If sysadmins have determined it's a UV problem can they point to a process ID or a user? Or is it because they can't find the source in Windows? -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Curt Stewart Sent: Friday, April 08, 2011 9:05 AM To: u2-users@listserver.u2ug.org Subject: [U2] Extreme slow down on Universe periodically Has anyone in the group experienced this issue and found a resolution? My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. This affects both existing processes (menu selections, data entry, reports) and logins. Their system admins (IBM) reports that it is a Universe or application problem. I haven't been able to identify any runaway processes or hogs. The log files that I've looked at haven't shed any light on the matter. Could there be a log file that I'm overlooking that might need to be cleared? But if it's an undersized file, why wouldn't it be a consistent problem? I'm hoping that the group might be able to suggest some things to look at and consider. Any and all suggestions would be appreciated. Environment Universe 10.2.10 Windows Server 2003 Virtual Machine (sorry don't know any details on this component) 34 user license, only about 15 active Thanks for your help. Curt Stewart TRI-SYS Consulting ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users CONFIDENTIAL INFORMATION: The information contained in this e-mail may be confidential and protected from general disclosure. If the recipient or reader of this e-mail is not the intended recipient or a person responsible to receive this e-mail for the intended recipient, please do not disseminate, distribute or copy it. If you received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. We will take immediate and appropriate action to see to it that this mistake is corrected.[*LD*] ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. This affects both existing processes (menu selections, data entry, reports) and logins. Their system admins (IBM) reports that it is a Universe or application problem. Curt, We have been experiencing slowdowns on some servers residing on virtual machines. So far, we are not certain, but we are beginning to suspect that we are overloading the network card on our virtual machine. We often see slowness on a web server for example and when we check resources it looks OK, but pings are slow and sometimes time out the request. Frequently, when we see the slowness on the guests it is followed by our Domino server (also on the VM) becoming unresponsive. This made us think network since the Domino server is very network intensive. This may turn out to not be the case, but it is something to consider. Charles Shaffer Senior Analyst NTN-Bower Corporation ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
Use sysinternal tools and watch IO delta bytes, you can also trace CPU utilization. hp On Fri, Apr 8, 2011 at 9:04 AM, Curt Stewart cstew...@tri-sysconsulting.com wrote: Has anyone in the group experienced this issue and found a resolution? My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. This affects both existing processes (menu selections, data entry, reports) and logins. Their system admins (IBM) reports that it is a Universe or application problem. I haven’t been able to identify any runaway processes or hogs. The log files that I’ve looked at haven’t shed any light on the matter. Could there be a log file that I’m overlooking that might need to be cleared? But if it’s an undersized file, why wouldn’t it be a consistent problem? I’m hoping that the group might be able to suggest some things to look at and consider. Any and all suggestions would be appreciated. Environment Universe 10.2.10 Windows Server 2003 Virtual Machine (sorry don’t know any details on this component) 34 user license, only about 15 active Thanks for your help. Curt Stewart TRI-SYS Consulting ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- *hp* ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
We have had that happen when the unix load average increased, and there were emails going out. sendmail stops at a certain load average, and kept spawning off more and more processes, increasing the load average even more. But, when it slows down, the easist thing is to drop to unix, and run the top command to show you the processes that are using cpu usage. George -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users- boun...@listserver.u2ug.org] On Behalf Of Curt Stewart Sent: Friday, April 08, 2011 9:05 AM To: u2-users@listserver.u2ug.org Subject: [U2] Extreme slow down on Universe periodically Has anyone in the group experienced this issue and found a resolution? My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. This affects both existing processes (menu selections, data entry, reports) and logins. Their system admins (IBM) reports that it is a Universe or application problem. I haven't been able to identify any runaway processes or hogs. The log files that I've looked at haven't shed any light on the matter. Could there be a log file that I'm overlooking that might need to be cleared? But if it's an undersized file, why wouldn't it be a consistent problem? I'm hoping that the group might be able to suggest some things to look at and consider. Any and all suggestions would be appreciated. Environment Universe 10.2.10 Windows Server 2003 Virtual Machine (sorry don't know any details on this component) 34 user license, only about 15 active Thanks for your help. Curt Stewart TRI-SYS Consulting ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
Curt: Try installing ProcessExplorer from Microsoft's SysInternals site... http://technet.microsoft.com/en-us/sysinternals/bb896653 This has solved a number of similar problems I've had with clients in the past. Most of these problems I trace down to a service that starts and is running with U2 slows down. Process explorer can report unusual CPU or other resource activity. HTH, Bill - Original Message - *From:* cstew...@tri-sysconsulting.com *To:* u2-users@listserver.u2ug.org *Date:* 4/8/2011 6:04 AM *Subject:* [U2] Extreme slow down on Universe periodically Has anyone in the group experienced this issue and found a resolution? My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. This affects both existing processes (menu selections, data entry, reports) and logins. Their system admins (IBM) reports that it is a Universe or application problem. I haven't been able to identify any runaway processes or hogs. The log files that I've looked at haven't shed any light on the matter. Could there be a log file that I'm overlooking that might need to be cleared? But if it's an undersized file, why wouldn't it be a consistent problem? I'm hoping that the group might be able to suggest some things to look at and consider. Any and all suggestions would be appreciated. Environment Universe 10.2.10 Windows Server 2003 Virtual Machine (sorry don't know any details on this component) 34 user license, only about 15 active Thanks for your help. Curt Stewart TRI-SYS Consulting ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
On 08/04/11 14:04, Curt Stewart wrote: Has anyone in the group experienced this issue and found a resolution? Any and all suggestions would be appreciated. Environment Universe 10.2.10 Windows Server 2003 Virtual Machine (sorry don’t know any details on this component) 34 user license, only about 15 active This is an old problem, that might well be resolved on 10.2, but do you have any rogue uvtelnets? We used to get that - they'd get disconnected somehow from their user, and then grab 100% cpu. Kill them, and things went back to normal (we had to crash them, they wouldn't kill, by attaching the debugger or something). Charles mentioned the network. Are your network buffers too large? I'm beginning to pick up from my reading on a hardware/software mismatch at the TCP level, and network slowdown (and catastrophic failure) are the result. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
Make certain that no one has decided that that server needs periodic virus scan checks. The database is seen as one very large file, and each change to it triggers the I've changed flag which triggers virus scanning. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
From: Curt Stewart My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. Curt, one of our clients has a large site where this has been a chronic issue for about a year. They've done all the exercises with process monitors, network monitors, etc. Rocket has been all over this but I'm not sure they have a solid resolution. I'll link you guys up off-list so that you may be able to benefit from their experience. Best, T Tony Gravagno Nebula Research and Development TG@ remove.pleaseNebula-RnD.com remove.pleaseNebula-RnD.com/blog Visit PickWiki.com! Contribute! http://Twitter.com/TonyGravagno ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Extreme slow down on Universe periodically
I'm sure you'll already have this in mind but I see the slow down as a symptom that can be caused by any number of things so even though I'm sure collaboration will be very helpful, it's not guaranteed that the same problem would exist in both locations. What may have passed inspection at your clients site may be the cause at Curt's client site. But you guys already knew that, I'm sure, so I'm just stating the obvious. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Friday, April 08, 2011 3:34 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Extreme slow down on Universe periodically From: Curt Stewart My client is reporting the periodic slow down of their system. It slows down to almost the point of a hang. Curt, one of our clients has a large site where this has been a chronic issue for about a year. They've done all the exercises with process monitors, network monitors, etc. Rocket has been all over this but I'm not sure they have a solid resolution. I'll link you guys up off-list so that you may be able to benefit from their experience. Best, T Tony Gravagno Nebula Research and Development TG@ remove.pleaseNebula-RnD.com remove.pleaseNebula-RnD.com/blog Visit PickWiki.com! Contribute! http://Twitter.com/TonyGravagno ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
