How are you doing backups? What OS?
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Maglinger, Paul
Sent: Wednesday, October 14, 2015 11:25 AM
To: New Exchange List ([email protected])
Subject: [Exchange] Microsoft Exchange RPC Client Access runaway
I'm hoping someone can give me some insight on this.
Running Update Rollup 6 Exchange Server 2010 SP2 on the mailbox servers set up
as a DAG.
For the past couple of weeks I'm arriving in the morning (usually on Wednesdays
and Fridays) seeing the Microsoft Exchange RPC Client Access service with 80+
cpu utilization. This generally starts sometime between 4 and 4:30am. I'm not
showing any problems in the event logs during that time. The only going on
around that time is the database health checks at around 4:44am and they all
are passing. When people start showing up and logging at around 7:30am I start
getting:
Log Name: Application
Source: VSS
Date: 10/14/2015 7:20:06 AM
Event ID: 11
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: COMSTAR1.scvl.com
Description:
Volume Shadow Copy Service information: The COM Server with CLSID
{e579ab5f-1cc4-44b4-bed9-de0991ff0623} and name IVssCoordinatorEx2 cannot be
started. Most likely the CPU is under heavy load. [0x80080005, Server execution
failed]
Log Name: Application
Source: VSS
Date: 10/14/2015 7:20:07 AM
Event ID: 8193
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: COMSTAR1.scvl.com
Description:
Volume Shadow Copy Service error: Unexpected error calling routine
CoCreateInstance. hr = 0x80080005, Server execution failed.
That's obvious. And later on, and I believe because the RPC service is banging
the CPU so hard, I start getting these for each of the databases:
Log Name: Application
Source: MSExchangeRepl
Date: 10/14/2015 8:03:23 AM
Event ID: 2153
Task Category: Service
Level: Error
Keywords: Classic
User: N/A
Computer: COMSTAR1.scvl.com
Description:
The log copier was unable to communicate with server 'COMSTAR2.scvl.com'. The
copy of database 'HQMBDB\COMSTAR1' is in a disconnected state. The
communication error was: An error occurred while communicating with server
'COMSTAR2.scvl.com'. Error: Unable to read data from the transport connection:
An existing connection was forcibly closed by the remote host. The copier will
automatically retry after a short delay.
And these for various RPC counters when I stop the service:
Log Name: Application
Source: MSExchange Common
Date: 10/14/2015 8:14:13 AM
Event ID: 106
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: COMSTAR1.scvl.com
Description:
Performance counter updating error. Counter name is Client: Foreground RPCs
succeeded, category name is MSExchange RpcClientAccess. Optional code: 3.
Exception: The exception thrown is : System.InvalidOperationException: The
requested Performance Counter is not a custom counter, it has to be initialized
as ReadOnly.
After restarting the service everything is fine.
Backups start at 7:00pm and finish up by 10pm. The only scheduled tasks that
run anywhere near that time is the AitAgent at 2:30am.
The first time it happened I did some Googling and ran a
PublicFolderDatabaseRepairRequest and MailboxRepairRequest. They found issues
on the public folders and I corrected them. Nothing was found with the
mailboxes. I ran the checks again today and can't find anything wrong.
Additional searching came up with stopping and placing the Exchange Search
Indexer and Search services in manual, rebooting the server and see if the
problem occurs again. If not, delete and rebuild the catalog index. The
article I found said that they ran without the search services for several days
and the users couldn't use search, and they were okay with that. Not sure how
much flak I'd get for doing this, so before going down this road I wanted to
get some input from you guys. Anyone run across this and care to share past
experience, or offer another suggestion?
Thanks,
Paul
