Great response, thanks Kevin!
From: [email protected] [mailto:[email protected]] On
Behalf Of Kevin Holman
Sent: Friday, November 24, 2017 5:32 PM
To: [email protected]
Subject: [msmom] RE: SCOM Monitor Snapshot synchronization state failing
Snapshot synchronization runs once per day, and is a complete resync of the
management group's configuration snapshot.
You can review the snapshot history in the SQL logs:
--Config Service Snapshot history
SELECT * FROM cs.workitem
WHERE WorkItemName like '%snap%'
ORDER BY WorkItemRowId DESC
It is not abnormal for snapshot to fail. It is "sensitive" and will often fail
due to changes in config coming in, or because of blocking/deadlocks.
Occasionally it will just time out - but you will normally see the timeout
happen quickly.
I rarely see the need to adjust the config service configuration settings in
that file from defaults, even in the largest clients. Most of the time this is
a band-aid to a bigger issue.
The biggest cause of snapshot failures is running Windows Server 2012 (non-R2)
or older, and needing to have .NET 4.5.1 or later applied. If you have the
default .NET 4.5, that will lead to timeout failures.
The good thing about snapshot, is that when it fails, it USUALLY kicks back off
again and eventually runs with success. So it really isn't a concern, until
you go more than 24 hours without a success. The SQL log above keeps two days
retention by default.
If you want to kick it off manually, simply run (in SQL):
EXEC SnapshotSynchronizationForce
Then run the query above again and you should see a status of "1" for running.
20 = success. 10 = failed.
From: [email protected]<mailto:[email protected]>
[mailto:[email protected]] On Behalf Of Gareth Miles
Sent: Friday, November 24, 2017 4:19 AM
To: [email protected]<mailto:[email protected]>
Subject: [msmom] RE: SCOM Monitor Snapshot synchronization state failing
Hi
The monitor has now come healthy, I assume one of the fixes I've tried has
worked.
Any info on how to effectively trouble shoot this fault would be appreciated.
As a side note, I still have the strange 29180 event, any ideas?
'The description for Event ID 29180 from source OpsMgr Management Configuration
cannot be found. Either the component that raises this event is not installed
on your local computer or the installation is corrupted. You can install or
repair the component on the local computer.'
Thanks
Gareth
From: Gareth Miles
Sent: Friday, November 24, 2017 9:41 AM
To: [email protected]<mailto:[email protected]>
Subject: SCOM Monitor Snapshot synchronization state failing
Hi Folks
I'm hoping someone is able to help as I'm at a loss.
I have a SCOM SP1 management group on UR10.
This started failing yesterday, there's been no new MP's imported, the last new
MP was imported on the 8th November and was an upgrade to SQL 2016.
My investigations started with the monitor 'Snapshot synchronization state'
failing with the below.
Yesterday we had the last LastSuccessfulExecutionAgeSeconds at about 90000,
it's currently at 162855
< DataItem type ="
Microsoft.SystemCenter.ManagementConfiguration.WorkItemStatsDataItem " time ="
2017-11-24T06:23:15.2658679Z " sourceHealthServiceId ="
5b1e40cd-7f3a-0e39-adff-75fb47884957 " >
< WorkItemName > SnapshotSynchronization </ WorkItemName >
< LastSuccessfulExecutionAgeSeconds > 162855 </
LastSuccessfulExecutionAgeSeconds >
< LastErrorMessage >
Microsoft.EnterpriseManagement.ManagementConfiguration.DataAccessLayer.DataAccessException:
Data access operation failed at
Microsoft.EnterpriseManagement.ManagementConfiguration.DataAccessLayer.DataAccessOperation.ExecuteSynchronously(Int32
timeoutSeconds, WaitHandle stopWaitHandle) at
Microsoft.EnterpriseManagement.ManagementConfiguration.SqlConfigurationStore.ConfigurationStore.ExecuteOperationSynchronously(IDataAccessConnectedOperation
operation, String operationName) at
Microsoft.EnterpriseManagement.ManagementConfiguration.SqlConfigurationStore.ConfigurationStore.StartSnapshot()
at
Microsoft.EnterpriseManagement.ManagementConfiguration.Engine.SnapshotSynchronizationWorkItem.StartSnapshot()
at
Microsoft.EnterpriseManagement.ManagementConfiguration.Engine.SnapshotSynchronizationWorkItem.ExecuteSharedWorkItem()
at
Microsoft.EnterpriseManagement.ManagementConfiguration.Interop.SharedWorkItem.ExecuteWorkItem()
----------------------------------- System.Data.SqlClient.SqlException
(0x80131904): Sql execution failed. Error 50000, Level 16, State 1, Procedure
SnapshotSynchronizationStart, Line 49, Message: Failed to set configuration
space lock. Other process uses config space at
System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean
breakConnection, Action`1 wrapCloseInAction) at
System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject
stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at
System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand
cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler,
TdsParserStateObject stateObj, Boolean& dataReady) at
System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds,
RunBehavior runBehavior, String resetOptionsString) at
System.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader() at
System.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult
asyncResult) at
System.Data.SqlClient.SqlCommand.EndExecuteNonQuery(IAsyncResult asyncResult)
at
Microsoft.EnterpriseManagement.ManagementConfiguration.DataAccessLayer.NonQuerySqlCommandOperation.SqlCommandCompleted(IAsyncResult
asyncResult) ClientConnectionId:015f7eef-0385-4db2-aaa0-196f264e5a66 </
LastErrorMessage >
</ DataItem >
Looking at this post,
https://blogs.technet.microsoft.com/momteam/2013/01/29/support-tip-config-service-deltasynchronization-process-fails-with-timeout-exception/<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblogs.technet.microsoft.com%2Fmomteam%2F2013%2F01%2F29%2Fsupport-tip-config-service-deltasynchronization-process-fails-with-timeout-exception%2F&data=02%7C01%7Ckevin.holman%40microsoft.com%7Cca33caeb3700487ba9a108d53328a891%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636471172097466155&sdata=L%2BObj4CU8Q7CKPwS6vfce45d%2F4s8nlyAG3lpDamCBI0%3D&reserved=0>,
I already had these,
<OperationTimeout DefaultTimeoutSeconds= & <Operation
Name="GetEntityChangeDeltaList" TimeoutSeconds= , set to 500, I increased them
to 900, but that's 15 minutes and seems excessive.
The monitor hasn't cleared after these changes.
To confirm nothing was stuck in limbo as it were, I've restarted all management
servers, and both SQL nodes hosting the DB instances.
I have 6 management servers, 5 are showing as successful with 29180 events for
the following
OpsMgr Management Configuration Service successfully executed
'ConfigStoreStatsUpdate' engine work item
OpsMgr Management Configuration Service successfully executed 'AgentAssignment'
engine work item
OpsMgr Management Configuration Service successfully executed 'SiteAssignment'
engine work item
OpsMgr Management Configuration Service successfully executed
'DirtyNotification' engine work item
OpsMgr Management Configuration Service successfully executed
'ConfigStoreGroom' engine work item
OpsMgr Management Configuration Service successfully executed
'AgentPoolAssignment' engine work item
OpsMgr Management Configuration Service successfully executed
'DeltaSynchronization' engine work item
OpsMgr Management Configuration Service successfully executed
'ConfigStoreIndexOptimize' engine work item
OpsMgr Management Configuration Service successfully executed
'LocalHealthServiceDirtyNotification' engine work item
OpsMgr Management Configuration Service successfully executed 'GetNextWorkItem'
engine work item
The one of the management servers is showing 29180 events for each of the above
but with the following
The description for Event ID 29180 from source OpsMgr Management Configuration
cannot be found. Either the component that raises this event is not installed
on your local computer or the installation is corrupted. You can install or
repair the component on the local computer.
If the event originated on another computer, the display information had to be
saved with the event.
The following information was included with the event:
OpsMgr Management Configuration Service
ConfigStoreIndexOptimize
The handle is invalid
I've run a repair on the management server install, which hasn't helped.
Does anyone have any other suggestions?
Kind Regards
Gareth Miles
SCOM Specialist
Phone: +27 31 580 1582
Mobile: +27 83 648 8459
Email: [email protected]<mailto:[email protected]>
Web:
www.derivco.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.derivco.com&data=02%7C01%7Ckevin.holman%40microsoft.com%7Cca33caeb3700487ba9a108d53328a891%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636471172097466155&sdata=31Qkzwu4kZiYKnvdpZQmLOW%2Bdtr9idp8FmXVURF6H4M%3D&reserved=0>
The information transmitted is intended only for the person or entity to which
it is addressed and may contain confidential and/or privileged material. Any
review, retransmission, dissemination or other use of, or taking of any action
in reliance upon, this information by persons or entities other than the
intended recipient is prohibited. If you received this in error, please contact
the sender and delete the material from any computer. Furthermore, the
information contained in this message, and any attachments thereto, is for
information purposes only and may contain the personal views and opinions of
the author, which are not necessarily the views and opinions of the company.
The information transmitted is intended only for the person or entity to which
it is addressed and may contain confidential and/or privileged material. Any
review, retransmission, dissemination or other use of, or taking of any action
in reliance upon, this information by persons or entities other than the
intended recipient is prohibited. If you received this in error, please contact
the sender and delete the material from any computer. Furthermore, the
information contained in this message, and any attachments thereto, is for
information purposes only and may contain the personal views and opinions of
the author, which are not necessarily the views and opinions of the company.
