Monday, January 7, 2013

Monitoring for WMI corruption issues and rebuilding WMI

Background
  • WMI is a Windows OS component that is present on every Windows server and PC
  • WMI corruption can cause failures that include failure to apply Group Policy
  • Some organizations rely upon Group Policy to secure servers, secure group membership to groups with elevated rights, and to provide the working environment for interactive Remote Desktop Services users
  • WMI failures could lead to service outages or security issues

Symptoms
  • See "Events to monitor below"
  • Group Policy does not apply, including for interactive users
  • On the security tab in WMIMGMT.MSC, WMI classes do not display correctly
  • "Not found" error when trying to connect to the WMI namespace using WBEMTEST.EXE
  • Network adapter list is blank when configuring Session Directory (Windows 2003)

Events to monitor
  • Event ID 43
    • Application log; Microsoft-Windows-WMI; Event ID 43; Windows Management Instrumentation ADAP failed to connect to namespace (…) with the following error (…)
    • This indicates the WMI namespace cannot be contacted
    • In the case of a recent outage, this failure was because the WMI namespace was corrupt
  • Event ID 10
    • Application log; Microsoft-Windows-WMI; Event ID 10; Event filter with query (…) could not be reactivated in namespace (…)
    • This is a “symptom” event that indicates a failure to query WMI. In the case of a recent outage, WMI the failure of this query indicated WMI was corrupt
  • Event ID 1104
    • "Windows was unable to read the Windows Management Instrumentation (WMI) filter information associated with the Group Policy object..."
    • Indicates a failure to query WMI
  • Event ID 1090 
    • Windows failed to record Resultant Set of Policy (RSoP) information, which describes the scope of Group Policy objects applied to the computer or user. This could be caused by Windows Management Instrumentation (WMI) service being disabled, stopped, or other WMI errors. Group Policy settings successfully applied to the computer or user; however, management tools may not report accurately.
    • Indicates a failure to query WMI

To resolve
  • Rebuild the WMI repository
  • Disable Resultant Set of Policy logging
  • Find other products that write to WMI and eliminate, if possible

Rebuild the WMI repository
Notes about the WMI rebuild:
  • Be careful when importing MOFs. WMI will auto-recover much of the original WMI namespace at start. Importing MOFs may not be necessary, and may restore the issue
  • I have experienced repeat issues on a server after performing a "salvage." Performing a rebuild has had consistently good results 


Disable Resultant Set of Policy logging
Resultant Set of Policy logging writes information to the WMI database for each user who logs on interactively. This causes the WMI database to grow, and can cause WMI database corruption.

Windows 2003
Computer Configuration > Administrative Templates > System > Group Policy
Turn off Resultant Set of Policy logging

Windows 2008
Computer Configuration > Policies > Administrative Templates > System > Group Policy
Turn off Resultant Set of Policy logging


Script to rebuild WMI
@echo off

   cls
   Echo.
   Echo This script will delete the current WMI repository and rebuild it
   Echo Deleting and rebuilding the WMI repository can cause impact
   Echo.
   Echo See this article for details
   Echo http://blogs.technet.com/b/askperf/archive/2009/04/13/wmi-rebuilding-the-wmi-repository.aspx
   echo.
   pause

:Start
   call :StopService WSRM
   call :StopService tmlisten
   call :StopService iphlpsvc
   call :StopService winmgmt
   call :RenameWMI
   call :StartService winmgmt
   call :StartService iphlpsvc
   call :StartService tmlisten
   call :StartService WSRM
   call :ImportMOF
   goto :End


:RenameWMI
   Echo.
   Echo Renaming the WMI repository folder
   for /f "tokens=1-8 delims=:/. " %%a in ('echo %date% %time%') do set FileExtension=%%d%%b%%c%%e%%f%%g
   ren "C:\WINDOWS\system32\wbem\Repository" Repository.%FileExtension%.old

   Echo.
   Echo Registering WMI DLLs
   cd /d %windir%\system32\wbem
   for /f %%s in ('dir /b /s *.dll') do echo %%s&regsvr32 /s %%s

   if /i not exist %windir%\SysWOW64\wbem goto :EOF
   cd /d %windir%\SysWOW64\wbem
   for /f %%s in ('dir /b /s *.dll') do echo %%s&regsvr32 /s %%s

   goto :EOF


:ImportMOF

   Echo.
   Echo Importing WMI MOF and MFL files
   Echo (It's normal for this to take a few minutes)
   Echo.

   cd /d %windir%\system32\wbem
   for /f "delims=" %%s in ('dir /s /b *.mof *.mfl') do echo %%s&mofcomp "%%s"
 
   if /i not exist %windir%\SysWOW64\wbem goto :ImportMOFNext
   cd /d %windir%\SysWOW64\wbem
   for /f "delims=" %%s in ('dir /s /b *.mof *.mfl') do echo %%s&mofcomp "%%s" 
:ImportMOFNext
   Echo.
   Echo MOF and MFL file import complete
   Echo Verify administrative consoles for installed applications and services
   Echo.
   goto :EOF


:StopService
   Echo.
   Echo Disabling and stopping the %1 service
   sc config %1 start= disabled
   if /i {%errorlevel%}=={9009} echo SC tool not installed or not available here. Exiting...&goto :End

   net stop %1 /y
   sc query %1 | find /i "running"
   if /i {%errorlevel%}=={0} echo Couldn't stop the %1 service. Exiting...&goto :End

   goto :EOF


:StartService
   Echo.
   Echo Starting %1
   sc config %1 start= demand
   net start %1
   sc config %1 start= auto
   goto :EOF




:End
   Echo Script ran to completion
   pause


:EOF


Event ID Detail
Event ID 43
Log Name:      Application
Source:        Microsoft-Windows-WMI
Date:          11/1/2012 9:25:58 PM
Event ID:      43
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      
Description:
Windows Management Instrumentation ADAP failed to connect to namespace \\.\root\cimv2 with the following error 0x80041002

Event ID 10
Log Name:      Application
Source:        Microsoft-Windows-WMI
Date:          11/1/2012 9:23:01 PM
Event ID:      10
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer: 
Description:
Event filter with query "SELECT * FROM __InstanceModificationEvent WITHIN 60 WHERE TargetInstance ISA "Win32_Processor" AND TargetInstance.LoadPercentage > 99" could not be reactivated in namespace "//./root/subscription" because of error 0x8004100e. Events cannot be delivered through this filter until the problem is corrected.

Event ID 10
Log Name:      Application
Source:        Microsoft-Windows-WMI
Date:          11/1/2012 9:23:01 PM
Event ID:      10
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      
Description:
Event filter with query "select * from MSFT_SCMEventLogEvent" could not be reactivated in namespace "//./root/subscription" because of error 0x8004100e. Events cannot be delivered through this filter until the problem is corrected.

Event ID 1104
Log Name:      System
Source:        Microsoft-Windows-GroupPolicy
Date:          1/7/2013 2:09:47 PM
Event ID:      1104
Task Category: None
Level:         Warning
Keywords:     
User:          USERNAME
Computer:      SERVERNAME.DOMAINNAME.COM
Description:
Windows was unable to read the Windows Management Instrumentation (WMI) filter information associated with the Group Policy object cn={D2F03D3C-A39F-4F9D-AD58-5793B8C82DA9},cn=policies,cn=system,DC=DOMMAINNAME,DC=com.This may be caused by a deleted WMI Filter defined in the domain that is still in use by Group Policy objects. Group Policy settings for this Group Policy object will not be enforced. Other Group Policy objects may still apply. Windows will attempt to retrieve this information at the next policy cycle. This speciffic problem may be resolved by identifying all GPOs that reference the WMI filter and removing the references. Contact an administrator if this event recurs for several hours.

Event ID 1090
Log Name:      System
Source:        Microsoft-Windows-GroupPolicy
Date:          1/7/2013 2:09:50 PM
Event ID:      1090
Task Category: None
Level:         Warning
Keywords:     
User:          USERNAME
Computer:      SERVERNAME.DOMAINNAME.COM
Description:
Windows failed to record Resultant Set of Policy (RSoP) information, which describes the scope of Group Policy objects applied to the computer or user. This could be caused by Windows Management Instrumentation (WMI) service being disabled, stopped, or other WMI errors. Group Policy settings successfully applied to the computer or user; however, management tools may not report accurately.