Veritas BackupExec 2014 R2 to tape. Windows 2008 R2 Enterprise. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Michael B. Smith Sent: Wednesday, October 14, 2015 4:44 PM To: [email protected] Subject: [Exchange] RE: Microsoft Exchange RPC Client Access runaway
How are you doing backups? What OS? -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Maglinger, Paul Sent: Wednesday, October 14, 2015 11:25 AM To: New Exchange List ([email protected]) Subject: [Exchange] Microsoft Exchange RPC Client Access runaway I'm hoping someone can give me some insight on this. Running Update Rollup 6 Exchange Server 2010 SP2 on the mailbox servers set up as a DAG. For the past couple of weeks I'm arriving in the morning (usually on Wednesdays and Fridays) seeing the Microsoft Exchange RPC Client Access service with 80+ cpu utilization. This generally starts sometime between 4 and 4:30am. I'm not showing any problems in the event logs during that time. The only going on around that time is the database health checks at around 4:44am and they all are passing. When people start showing up and logging at around 7:30am I start getting: Log Name: Application Source: VSS Date: 10/14/2015 7:20:06 AM Event ID: 11 Task Category: None Level: Error Keywords: Classic User: N/A Computer: COMSTAR1.scvl.com Description: Volume Shadow Copy Service information: The COM Server with CLSID {e579ab5f-1cc4-44b4-bed9-de0991ff0623} and name IVssCoordinatorEx2 cannot be started. Most likely the CPU is under heavy load. [0x80080005, Server execution failed] Log Name: Application Source: VSS Date: 10/14/2015 7:20:07 AM Event ID: 8193 Task Category: None Level: Error Keywords: Classic User: N/A Computer: COMSTAR1.scvl.com Description: Volume Shadow Copy Service error: Unexpected error calling routine CoCreateInstance. hr = 0x80080005, Server execution failed. That's obvious. And later on, and I believe because the RPC service is banging the CPU so hard, I start getting these for each of the databases: Log Name: Application Source: MSExchangeRepl Date: 10/14/2015 8:03:23 AM Event ID: 2153 Task Category: Service Level: Error Keywords: Classic User: N/A Computer: COMSTAR1.scvl.com Description: The log copier was unable to communicate with server 'COMSTAR2.scvl.com'. The copy of database 'HQMBDB\COMSTAR1' is in a disconnected state. The communication error was: An error occurred while communicating with server 'COMSTAR2.scvl.com'. Error: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. The copier will automatically retry after a short delay. And these for various RPC counters when I stop the service: Log Name: Application Source: MSExchange Common Date: 10/14/2015 8:14:13 AM Event ID: 106 Task Category: General Level: Error Keywords: Classic User: N/A Computer: COMSTAR1.scvl.com Description: Performance counter updating error. Counter name is Client: Foreground RPCs succeeded, category name is MSExchange RpcClientAccess. Optional code: 3. Exception: The exception thrown is : System.InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly. After restarting the service everything is fine. Backups start at 7:00pm and finish up by 10pm. The only scheduled tasks that run anywhere near that time is the AitAgent at 2:30am. The first time it happened I did some Googling and ran a PublicFolderDatabaseRepairRequest and MailboxRepairRequest. They found issues on the public folders and I corrected them. Nothing was found with the mailboxes. I ran the checks again today and can't find anything wrong. Additional searching came up with stopping and placing the Exchange Search Indexer and Search services in manual, rebooting the server and see if the problem occurs again. If not, delete and rebuild the catalog index. The article I found said that they ran without the search services for several days and the users couldn't use search, and they were okay with that. Not sure how much flak I'd get for doing this, so before going down this road I wanted to get some input from you guys. Anyone run across this and care to share past experience, or offer another suggestion? Thanks, Paul
