tag:blogger.com,1999:blog-189895862024-02-20T11:00:12.352+01:00Windbg by Volker von EinemA collection of useful items to deal with windbg.
For me and possibly for others...Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.comBlogger69125tag:blogger.com,1999:blog-18989586.post-21548179194136438102009-12-04T07:54:00.001+01:002009-12-04T10:04:30.817+01:00Displaying dynamically allocated c arrays<p>I just analyzed a crash dump were I needed to investigate a dynamically allocated a array.</p> <p>We have a declaration:</p> <p>CString **ids;</p> <p>the allocation</p> <p>ids = new CString*[size];</p> <p>Given a size of 8 you can dump the content of the string array with dt -a<size> <address> <type>:</p> <p>0:000> dv ids <br />            ids = 0x06314ec0</p> <p>0:000> dt -a8 0x06314ec0 CString* <br />[0] @ 06314ec0 <br />--------------------------------------------- <br />0x0633f520 <br />   +0x000 m_pszData        : 0x062b7d58  "string 1" </p> <p>[1] @ 06314ec4 <br />--------------------------------------------- <br />0x062b7278 <br />   +0x000 m_pszData        : 0x785039c0  "" </p> <p>[2] @ 06314ec8 <br />--------------------------------------------- <br />0x0638b8b0 <br />   +0x000 m_pszData        : 0x06373958  "string 2" </p> <p>[3] @ 06314ecc <br />--------------------------------------------- <br />0x06373b10 <br />   +0x000 m_pszData        : 0x06301dc8  "string 3" </p> <p>[4] @ 06314ed0 <br />--------------------------------------------- <br />0x06338780 <br />   +0x000 m_pszData        : 0x785039c0  "" </p> <p>[5] @ 06314ed4 <br />--------------------------------------------- <br />0x06350a00 <br />   +0x000 m_pszData        : 0x785039c0  "" </p> <p>[6] @ 06314ed8 <br />--------------------------------------------- <br />0x0637e2f8 <br />   +0x000 m_pszData        : 0x785039c0  "" </p> <p>[7] @ 06314edc <br />--------------------------------------------- <br />0x0633f598 <br />   +0x000 m_pszData        : 0x785039c0  "" </p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-56780384998129074822009-02-20T22:17:00.001+01:002009-02-20T22:18:58.176+01:00Windbg 6.11.1.402 is out<p>There's no support for <a href="http://voneinem-windbg.blogspot.com/2007/04/windbg-6750-released.html">integrated managed debugging</a>. Here's the link:</p> <p><a title="http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx#a" href="http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx#a">http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx#a</a></p> <p> </p> <blockquote> <p>Highlights in Version 6.11.1.402 <br />The following changes have been made to Debugging Tools for Windows: </p> <p>• Numerous bug fixes and documentation updates <br />• Numerous updates to improve 1394 debugging (see relnotes.txt for details) <br />• support using “.process /p …” in kd -kl, so you can see user mode memory in the appropriate process context (which means user mode stacks, !peb, etc.) </p></blockquote> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com4tag:blogger.com,1999:blog-18989586.post-17509652356911111422009-02-13T11:43:00.001+01:002009-02-13T11:43:32.442+01:00Another goody: uf /c /D and the peb lock<p>Remember my previous <a href="http://voneinem-windbg.blogspot.com/2009/02/keeping-pearl.html">post</a>?</p> <p>I was hunting a dead lock and found the good old 6.7.5.0 again worth keeping.</p> <p>But I didn't write about the dead lock cause itself. Now here it comes:</p> <p> </p> <p>The main thread was waiting for the peb lock:</p> <p>0:00> ~0 kb <br />ChildEBP RetAddr Args to Child <br />0012c72c 7c90e9c0 7c91901b 0000056c 00000000 ntdll!KiFastSystemCallRet <br />0012c730 7c91901b 0000056c 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc <br />0012c7b8 7c90104b 0097e4c0 7c910945 7c97e4c0 ntdll!RtlpWaitForCriticalSection+0x132 <br />0012c7c0 7c910945 7c97e4c0 ffffffff 00000000 ntdll!RtlEnterCriticalSection+0x46 <br />0012c800 7c80e2cf ffffffff 7c80e038 00000000 ntdll!RtlAcquirePebLock+0x28 <br />0012c868 7c80e00d 7c80e038 00020690 00000000 kernel32!BasepComputeProcessPath+0x5f <br />0012c8a8 7c801b83 00000000 00000000 0012d7a0 kernel32!BaseComputeProcessDllPath+0xe3 <br />0012c908 7c801d6e 7ffdfc00 00000000 00000000 kernel32!LoadLibraryExW+0x12f <br />0012c91c 7c801da4 06057954 00000000 00000000 kernel32!LoadLibraryExA+0x1f <br />0012c938 060b4823 06057954 7c911993 03f41ea8 kernel32!LoadLibraryA+0x94</p> <p> </p> <p>This lock was owned by managed thread number 6:</p> <p>0:006> k 100 <br />ChildEBP RetAddr  <br />0522f33c 7c9141e4 ntdll!RtlGetFullPathName_Ustr+0x653 <br />0522f364 7c80b86c ntdll!RtlGetFullPathName_U+0x33 <br />0522f37c 015da1c3 kernel32!GetFullPathNameW+0x1a <br />0522f3a4 79364aef CLRStub[StubLinkStub]@15da1c3 <br />0522f8f0 793644ae mscorlib_ni!System.IO.Path.NormalizePathFast(System.String, Boolean)+0x637 <br />017d2738 793821b0 mscorlib_ni!System.IO.Path.GetFullPathInternal(System.String)+0x2e <br />017d2738 793820f6 mscorlib_ni!System.Security.Util.StringExpressionSet.CanonicalizePath(System.String, Boolean)+0x90 <br />017d2738 79381dbe mscorlib_ni!System.Security.Util.StringExpressionSet.CreateListFromExpressions(System.String[], Boolean)+0x14a <br />017d2738 79381d16 mscorlib_ni!System.Security.Permissions.FileIOPermission.AddPathList(System.Security.Permissions.FileIOPermissionAccess, System.Security.AccessControl.AccessControlActions, System.String[], Boolean, Boolean, Boolean)+0x6a <br />0178b9ec 793a2014 mscorlib_ni!System.Security.Permissions.FileIOPermission..ctor(System.Security.Permissions.FileIOPermissionAccess, System.String)+0x42 <br />0522f978 04189669 mscorlib_ni!System.IO.FileSystemInfo.get_FullName()+0x58 <br />0177cdbc 041892b2 XYZ.DirectoryCleanup.PopulateAllFilesList(System.Collections.Generic.List`1<System.IO.FileInfo>, System.IO.DirectoryInfo)+0x69 <br />0522fa00 0418916f XYZ.DirectoryCleanup.CleanupThread()+0xaa <br />0522fa30 79407caa XYZ.DirectoryCleanup.CleanupThreadProc(System.Object)+0x67 <br />0179439c 79373ecd mscorlib_ni!System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(System.Object)+0x1a <br />0179439c 79407e18 mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)+0x81 <br />0522fa60 79407d90 mscorlib_ni!System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(System.Threading._ThreadPoolWaitCallback)+0x50 <br />1f7814b0 79e7c74b mscorlib_ni!System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(System.Object)+0x60 <br />0522fa84 79e7c6cc mscorwks!CallDescrWorker+0x33 <br />0522fb04 79f00eca mscorwks!CallDescrWorkerWithHandler+0xa3 <br />0522fb24 79f00e75 mscorwks!DispatchCallBody+0x1e <br />0522fb88 79f00f03 mscorwks!DispatchCallDebuggerWrapper+0x3d <br />0522fbbc 79f6e39d mscorwks!DispatchCallNoEH+0x51 <br />0522fc1c 79ef3207 mscorwks!QueueUserWorkItemManagedCallback+0x6c <br />0522fc30 79ef31a3 mscorwks!Thread::DoADCallBack+0x32a <br />0522fcc4 79ef30c3 mscorwks!Thread::ShouldChangeAbortToUnload+0xe3 <br />0522fd00 79ef4826 mscorwks!Thread::ShouldChangeAbortToUnload+0x30a <br />0522fd28 79ef48cf mscorwks!Thread::ShouldChangeAbortToUnload+0x33e <br />0522fd40 79f6e2d8 mscorwks!ManagedThreadBase::ThreadPool+0x13 <br />0522fda8 79f0202a mscorwks!ManagedPerAppDomainTPCount::DispatchWorkItem+0xdb <br />0522fdbc 79f021a0 mscorwks!ThreadpoolMgr::ExecuteWorkRequest+0xaf <br />0522fe14 79f95a2e mscorwks!ThreadpoolMgr::WorkerThreadStart+0x223 <br />0522ffb4 7c80b683 mscorwks!Thread::intermediateThreadProc+0x49 <br />0522ffec 00000000 kernel32!BaseThreadStart+0x37</p> <p>Now my question was: Does kernel32!GetFullPathNameW acquire a peb lock?</p> <p>The short answer is yes (in Windows XP but not in Vista).</p> <p>Here's the easy way to find out with uf /c /D (/D creates linked callee names for navigation of the call graph. Do you can click on the hyperlinks ;-) ):</p> <p><font face="Courier New">0:006> uf /c /D kernel32!GetFullPathNameW <br />kernel32!GetFullPathNameW (7c80b852) <br />  kernel32!GetFullPathNameW+0x14 (7c80b866): <br />    </font><font face="Courier New"><font color="#0080ff"><u>call to ntdll!RtlGetFullPathName_U (7c9141b1) <br /></u></font><font color="#000000"></font></font></p> <p><font face="Courier New"><font color="#000000">0:006> uf /c /D 0x7c9141b1 <br />ntdll!RtlGetFullPathName_U (7c9141b1) <br />  ntdll!RtlGetFullPathName_U+0xe (7c9141bf): <br />    <u><font color="#0080ff">call to ntdll!RtlInitUnicodeStringEx (7c9103a5)</font></u> <br />  ntdll!RtlGetFullPathName_U+0x2e (7c9141df): <br />    <font color="#0080ff"><u>call to ntdll!RtlGetFullPathName_Ustr (7c913b67)</u></font> <br /></font></font></p> <p><font face="Courier New"><font face="Courier New" color="#000000">0:006> uf /c /D 0x7c913b67 <br />ntdll!RtlGetFullPathName_Ustr (7c913b67) <br />  ntdll!RtlGetFullPathName_Ustr+0xa (7c913b71): <br />    <u><font color="#0080ff">call to ntdll!_SEH_prolog (7c90edc2)</font></u> <br />  ntdll!RtlGetFullPathName_Ustr+0xbe (7c913c19): <br />    <u><font color="#0080ff">call to ntdll!RtlIsDosDeviceName_Ustr (7c9139ed)</font></u> <br />  ntdll!RtlGetFullPathName_Ustr+0x176 (7c913c59): <br />    <u><font color="#0080ff">call to ntdll!RtlDetermineDosPathNameType_U (7c91399f)</font></u> <br />  ntdll!RtlGetFullPathName_Ustr+0x180 (7c913c63): <br /><strong><font size="3">    <u><font color="#0080ff">call to ntdll!RtlAcquirePebLock (7c91091d) </font><font color="#ff0000"><== Here you go</font></u></font></strong> <br />[...]</font></font></p> <p> </p> <p>Doing the same on Vista shows no PebLock</p> <p><font face="Courier New">0:003> uf /c /D kernel32!GetFullPathNameW <br />KERNEL32!GetFullPathNameW (77c60a0e) <br />  KERNEL32!GetFullPathNameW+0x14 (77c60a22): <br />    call to ntdll!RtlGetFullPathName_U (77b0ac1f) <br />0:003> uf /c /D 0x77b0ac1f <br />ntdll!RtlGetFullPathName_U (77b0ac1f) <br />  ntdll!RtlGetFullPathName_U+0xe (77b0ac2d): <br />    call to ntdll!RtlInitUnicodeStringEx (77b09b9f) <br />  ntdll!RtlGetFullPathName_U+0x2e (77b0ac4d): <br />    call to ntdll!RtlGetFullPathName_Ustr (77b0a01f) <br />0:003> uf /c /D 0x77b0a01f <br />ntdll!RtlGetFullPathName_Ustr (77b0a01f) <br />  ntdll!RtlGetFullPathName_Ustr+0x270 (77aaa47d): <br />    call to ntdll!RtlpReferenceCurrentDirectory (77b0aed5) <br />  ntdll!RtlGetFullPathName_Ustr+0x2a3 (77aaa497): <br />    call to ntdll!RtlpComputeBackupIndex (77b0af83) <br />  ntdll!RtlGetFullPathName_Ustr+0x2f9 (77ab0340): <br />    call to ntdll!RtlpReferenceCurrentDirectory (77b0aed5) <br />  ntdll!RtlGetFullPathName_Ustr+0x32b (77ab0366): <br />    call to ntdll!RtlUpcaseUnicodeChar (77ae0db2) <br />  ntdll!RtlGetFullPathName_Ustr+0x339 (77ab0374): <br />    call to ntdll!RtlUpcaseUnicodeChar (77ae0db2) <br />  ntdll!RtlGetFullPathName_Ustr+0x366 (77ab0386): <br />    call to ntdll!RtlpCheckRelativeDrive (77ab040c) <br />  ntdll!RtlGetFullPathName_Ustr+0x38c (77ab03ac): <br />    call to ntdll!RtlInitUnicodeString (77ae7e70) <br />  ntdll!RtlGetFullPathName_Ustr+0x3a9 (77ab03c9): <br />    call to ntdll!RtlQueryEnvironmentVariable_U (77ad4c19) <br />  ntdll!RtlGetFullPathName_Ustr+0x11e (77ad8154): <br />    call to ntdll!memmove (77ae8d80) <br />  ntdll!RtlGetFullPathName_Ustr+0x7 (77b0a026): <br />    call to ntdll!_SEH_prolog4_GS (77b08f60) <br />  ntdll!RtlGetFullPathName_Ustr+0xb0 (77b0a0c3): <br />    call to ntdll!RtlIsDosDeviceName_Ustr (77b09bf1) <br />  ntdll!RtlGetFullPathName_Ustr+0x15e (77b0a0e7): <br />    call to ntdll!memset (77ae90c0) <br />  ntdll!RtlGetFullPathName_Ustr+0x16a (77b0a0f3): <br />    call to ntdll!RtlDetermineDosPathNameType_Ustr (77b08ed3) <br />  ntdll!RtlGetFullPathName_Ustr+0x83e (77b0a27d): <br />    call to ntdll!RtlGetFullPathName_Ustr+0x84b (77b0a292) <br />  ntdll!RtlGetFullPathName_Ustr+0x141 (77b0a285): <br />    call to ntdll!_SEH_epilog4_GS (77b08fa8) <br />  ntdll!RtlGetFullPathName_Ustr+0x21b (77b0ae76): <br />    call to ntdll!RtlpReferenceCurrentDirectory (77b0aed5) <br />  ntdll!RtlGetFullPathName_Ustr+0x261 (77b0aea0): <br />    call to ntdll!RtlpComputeBackupIndex (77b0af83) <br />  ntdll!RtlGetFullPathName_Ustr+0xe9 (77b10822): <br />    call to ntdll!RtlpCheckDeviceName (77b2cf1c) <br />  ntdll!RtlGetFullPathName_Ustr+0x3f2 (77b1093d): <br />    call to ntdll!RtlInitUnicodeString (77ae7e70)</font></p> <p> </p> <p>Many guys are offending Vista for this and that (mainly UAC stuff which sometimes really sucks) but for me this is another prove that Vista behaves more stable than previous Windows versions.</p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-22786095882687606962009-02-12T11:45:00.002+01:002009-02-13T09:21:12.711+01:00Keeping the Pearl<p>I recently get a memory dump of a hanging process. So I opened it with <a href="http://voneinem-windbg.blogspot.com/2008/11/windbg-version-6103233-is-out-i-still.html">windbg 6.10.3.233</a></p> <p>The main thread was looking like this:</p> <p style="font-family: courier new;">0:00> ~0 kb <br />ChildEBP RetAddr Args to Child <br />0012c72c 7c90e9c0 7c91901b 0000056c 00000000 ntdll!KiFastSystemCallRet <br />0012c730 7c91901b 0000056c 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc <br />0012c7b8 7c90104b 0097e4c0 7c910945 7c97e4c0 ntdll!RtlpWaitForCriticalSection+0x132 <br />0012c7c0 7c910945 7c97e4c0 ffffffff 00000000 ntdll!RtlEnterCriticalSection+0x46 <br />0012c800 7c80e2cf ffffffff 7c80e038 00000000 ntdll!RtlAcquirePebLock+0x28 <br />0012c868 7c80e00d 7c80e038 00020690 00000000 kernel32!BasepComputeProcessPath+0x5f <br />0012c8a8 7c801b83 00000000 00000000 0012d7a0 kernel32!BaseComputeProcessDllPath+0xe3 <br />0012c908 7c801d6e 7ffdfc00 00000000 00000000 kernel32!LoadLibraryExW+0x12f <br />0012c91c 7c801da4 06057954 00000000 00000000 kernel32!LoadLibraryExA+0x1f <br />0012c938 060b4823 06057954 7c911993 03f41ea8 kernel32!LoadLibraryA+0x94</p> <p style="font-family: courier new;">Sieextpub's very usefull !critlist extension showed me were whose holding the critical section:</p> <p style="font-family: courier new;">0:000> !critlist <br />CritSec at 7c97e4c0. Owned by thread 6.<br />Waiting Threads: 0 21 <br />CritSec at 7c97c0d8. Owned by thread 21.<br />Waiting Threads: 22 </p> <p>SOS!Threads told me that it's a managed thread from the thread pool:</p> <p style="font-family: courier new;">0:006> !Threads <br />ThreadCount: 5 <br />UnstartedThread: 0 <br />BackgroundThread: 5 <br />PendingThread: 0 <br />DeadThread: 0 <br />Hosted Runtime: no <br /> PreEmptive GC Alloc Lock <br /> ID OSID ThreadOBJ State GC Context Domain Count APT Exception <br /> 0 1 ea4 001886e0 4220 Enabled 00000000:00000000 0019c7b8 1 STA System.Runtime.InteropServices.SEHException (0177d128) <br /> 2 2 2c0 001a6210 b220 Enabled 00000000:00000000 0019c7b8 0 MTA (Finalizer) <br /> 3 3 d7c 001efbd0 80a220 Enabled 00000000:00000000 0019c7b8 0 MTA (Threadpool Completion Port) <br /> 4 4 894 00223760 880b220 Enabled 00000000:00000000 0019c7b8 0 MTA (Threadpool Completion Port) <br /> 6 5 650 00224310 180b220 Enabled 017d2770:017d4608 0019c7b8 0 MTA (Threadpool Worker)</p> <p><span style="font-family:Terminal;">Now I wanted to get the Stack Dump for thread number 6:</span> <br /></p> <p style="font-family: courier new;">0:006> !CLRStack <br />OS Thread Id: 0x650 (6) <br />ESP EIP <br />0522f3bc 7c913da4 [NDirectMethodFrameStandalone: 0522f3bc] <br />0522f3d4 79364aef <br />0522f8f8 793644ae <br />0522f900 793821b0 <br />0522f90c 793820f6 <br />0522f928 79381dbe <br />0522f950 79381d16 <br />0522f968 793a2014 <br />0522f980 04189669 <br />0522f9a0 041892b2 <br />0522fa08 0418916f <br />0522fa38 79407caa <br />0522fa3c 79373ecd <br />0522fa54 79407e18 <br />0522fa68 79407d90 <br />0522fbf8 79e7c74b [GCFrame: 0522fbf8] </p> <p>Oops? OK - sometimes !DumpStack does a better job:</p> <p style="font-family: courier new;">0:006> !DumpStack <br />OS Thread Id: 0x650 (6) <br />Current frame: ntdll!RtlGetFullPathName_Ustr+0x653 <br />ChildEBP RetAddr Caller,Callee <br />0522f2d0 79e75002 mscorwks!FrameWithCookie<HelperMethodFrame>::FrameWithCookie<HelperMethodFrame>+0x1e, calling mscorwks!HelperMethodFrame::HelperMethodFrame <br />0522f318 7c91056d ntdll!RtlFreeHeap+0x647, calling ntdll!_SEH_epilog <br />0522f33c 7c9141e4 ntdll!RtlGetFullPathName_U+0x33, calling ntdll!RtlGetFullPathName_Ustr <br />0522f364 7c80b86c kernel32!GetFullPathNameW+0x1a, calling ntdll!RtlGetFullPathName_U <br />0522f37c 015da1c3 015da1c3 <br />0522f3a4 79364aef 79364aef, calling 796c00cc <br />0522f3c4 79364aef 79364aef, calling 796c00cc <br />0522f8f0 793644ae 793644ae, calling 793644b8 <br />0522f8f8 793821b0 793821b0, calling 79364480 <br />0522f904 793820f6 793820f6, calling 79382120 <br />0522f920 79381dbe 79381dbe, calling 79381fac <br />0522f934 79381d16 79381d16, calling 79381d54 <br />0522f95c 793a2014 793a2014, calling 79381cd4 <br />0522f978 04189669 04189669 <br />0522f994 041892b2 041892b2, calling 04189600 <br />0522fa00 0418916f 0418916f, calling 04189208 <br />0522fa30 79407caa 79407caa <br />0522fa34 79373ecd 79373ecd <br />0522fa48 79407e18 79407e18, calling 79373e4c <br />0522fa60 79407d90 79407d90, calling 79407dc8 <br />0522fa74 79e7c74b mscorwks!CallDescrWorker+0x33 <br />0522fa84 79e7c6cc mscorwks!CallDescrWorkerWithHandler+0xa3, calling mscorwks!CallDescrWorker <br />0522fb04 79f00eca mscorwks!DispatchCallBody+0x1e, calling mscorwks!CallDescrWorkerWithHandler <br />0522fb24 79f00e75 mscorwks!DispatchCallDebuggerWrapper+0x3d, calling mscorwks!DispatchCallBody <br />0522fb88 79f00f03 mscorwks!DispatchCallNoEH+0x51, calling mscorwks!DispatchCallDebuggerWrapper <br />0522fbbc 79f6e39d mscorwks!QueueUserWorkItemManagedCallback+0x6c, calling mscorwks!DispatchCallNoEH <br />0522fc1c 79ef3207 mscorwks!Thread::DoADCallBack+0x32a <br />0522fc30 79ef31a3 mscorwks!Thread::ShouldChangeAbortToUnload+0xe3, calling mscorwks!Thread::DoADCallBack+0x2db <br />0522fc58 79e744d0 mscorwks!Thread::EnterRuntimeNoThrow+0x94, calling ntdll!RtlSetLastWin32Error <br />0522fc5c 79e744d7 mscorwks!Thread::EnterRuntimeNoThrow+0x9b, calling mscorwks!_EH_epilog3 <br />0522fcc4 79ef30c3 mscorwks!Thread::ShouldChangeAbortToUnload+0x30a, calling mscorwks!Thread::ShouldChangeAbortToUnload+0x32 <br />0522fd00 79ef4826 mscorwks!Thread::ShouldChangeAbortToUnload+0x33e, calling mscorwks!Thread::ShouldChangeAbortToUnload+0x29d <br />0522fd28 79ef48cf mscorwks!ManagedThreadBase::ThreadPool+0x13, calling mscorwks!Thread::ShouldChangeAbortToUnload+0x319 <br />0522fd40 79f6e2d8 mscorwks!ManagedPerAppDomainTPCount::DispatchWorkItem+0xdb, calling mscorwks!ManagedThreadBase::ThreadPool <br />0522fd94 79f022c7 mscorwks!PerAppDomainTPCountList::GetAppDomainIndexForThreadpoolDispatch+0x35 <br />0522fda8 79f0202a mscorwks!ThreadpoolMgr::ExecuteWorkRequest+0xaf <br />0522fdbc 79f021a0 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x223, calling mscorwks!ThreadpoolMgr::ExecuteWorkRequest <br />0522fe14 79f95a2e mscorwks!Thread::intermediateThreadProc+0x49 <br />0522ffa0 79f95a1c mscorwks!Thread::intermediateThreadProc+0x37, calling mscorwks!_alloca_probe_16 <br />0522ffb4 7c80b683 kernel32!BaseThreadStart+0x37</p> <p> </p> <p>Not really better, eh?</p> <p>So I launched my <a href="http://voneinem-windbg.blogspot.com/2007/04/windbg-6750-released.html">good old 6.7.5.0</a> (which lives side by side with the latest windbg on my machine - thanks to xcopy) and surprise:</p> <p>[click image to enlarge]</p> <p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjemDJSbnHm0mIdw-Uf9xw8MajGPFlNAhvWvUudQW3xPMwyUDxgbTbIXwIEf1jbt_QEFHvhJb_vokTF-qQnFkAbSRBc5dvfDDsL3VOcWXuTyrmroj4qMpe-gm3HrT1EbN97oOJF/s1600-h/6750thePearcl%5B7%5D.png"><img style="border: 0px none ;" alt="6750thePearcl" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOc49zEK3ZFuFFwd5zos8im83Q-UFTOhm8Udw8jExtGCZDRSIbPX0PmuuVURcXBqzJYwbGxJ5TEnc3lFk_95t28gwFiYZoJwF8CFdXjtDMylp8Tg18ygqSLhrqgTnqne2P9NDA/?imgmax=800" width="644" border="0" height="484" /></a> </p> <p>The call stack window did a wonderful job and I even got pointed to the corresponding source code.</p> <p>As it is the same sos, a !DumpStack did not do the job in 6.7.5.0 either.</p> <p> </p> <p>So please Microsoft, give us back the this very very helpful feature of 6.7.5.0!<br />Or at least make SOS working like the stack window of 6.7.5.0.</p>Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com3tag:blogger.com,1999:blog-18989586.post-35784822589799180022009-02-10T08:41:00.001+01:002009-02-10T08:41:46.697+01:00Article about Remote Debugging<p><a href="http://blogs.msdn.com/ntdebugging/archive/tags/Mark/default.aspx">Mark?</a> has published a very good <a href="http://blogs.msdn.com/ntdebugging/archive/2009/02/09/remote-debugging-connecting-to-a-remote-stub-using-the-microsoft-debugging-tools-for-windows.aspx">article about Remote Debugging</a> on the <a href="http://blogs.msdn.com/ntdebugging/">Microsoft Advanced Windows Debugging and Troubleshooting Blog</a>. </p> <p>Enjoy!</p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-79294147294652371022008-12-18T14:50:00.002+01:002008-12-19T06:15:18.606+01:00Symbol Server Performance Improvement<p>Once you've fallen in love with your own symbol server you might discover bad performance the bigger the symbol store grows. <br />We've recently moved our private symbol server onto a new server and found in the symhttp.doc the following:</p> <p> </p> <blockquote> <p>Normally files are placed in a symbol store with a single tier directory structure in which a single subdirectory exists to store all versions of a certain filename. Such a tree may look like this</p> <p>c:\symstore\ntdll.dll\</p> <p>c:\symstore\ntdll.pdb\</p> <p>c:\symstore\kernel32.dll\</p> <p>c:\symstore\kernel32.pdb\</p> <p><strong><span style="color:#ff0000;">However if you are going to store a massive amount of filenames, you may prefer to use the two-tier structure. To do this, place a file called <i>index2.txt</i> in the root of <i>c:\symstore</i></span></strong>. The contents of the file are of no importance. This would result in a tree that looks like this</p> <p>c:\symstore\<b>nt</b>\ntdll.dll\</p> <p>c:\symstore\<b>nt</b>\ntdll.pdb\</p> <p>c:\symstore\<b>ke</b>\kernel32.dll\</p> <p>c:\symstore\<b>ke</b>\kernel32.pdb\</p> <p>The added layer of directories allows you to use DFS or directory junctions to manage and split up your files.</p> </blockquote> <p>Putting the empty index2.txt in the root is not really a task to mention, but what happens to all the files you have added over time? <br />As a software developer I tend to do things the lazy way. So rerunning symstore on the old configurations is something for those having killed father and mother. <br />Thanks to the power of <a href="http://en.wikipedia.org/wiki/Language_Integrated_Query">LINQ</a> and <a href="http://code.google.com/p/nconsoler/">NConsoler</a> I could write a handy <a href="http://vve.gmxhome.de/docs/MigrateSymbolStoreToIndex.zip">tool</a> in minutes that does the migration stuff for me (and you).<br /></p> <p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhypiuHGadidEWI5yj_y73vhjt704vv3FoM3n7o8Wri0P46LugfIird__ziBeFZDWj9HHXWfFOv_cXM1Y5YNvqGjdaUkivxFIgo7M1m-Eevf-p10gqqqUobRjGzUhnGI6exvjID/s1600-h/image%5B5%5D.png"><img style="border: 0px none ;" alt="image" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibfnKieuVj-72n6bbtYvMXfGNwigXBZH2qZmg62IN14kCajSLQhiJZizLAwwJHjJlMzZBO-505LigTkONLNZqbwA3gnvEcNUVgzcBUqO9KTNBqahNuD-PQBN2icNVOzK6K_2wi/?imgmax=800" border="0" height="318" width="644" /></a> </p> <p>Lay back and feel the speed ;-)</p><p><br /></p>[Please backup the symbol store before migration - I give no warranty!]Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com1tag:blogger.com,1999:blog-18989586.post-75108545821985287212008-11-24T16:23:00.001+01:002008-11-25T09:31:18.688+01:00Windbg version 6.10.3.233 is out (I'm still keeping 6.7.5.0)<p>As some blogs (<a href="http://shellexecute.wordpress.com/2008/11/23/windbg-new-release-6103233/">here</a> and <a href="http://www.dumpanalysis.org/blog/index.php/2008/11/24/windbg-release-6103233/">here</a>) already published there is a new version of windbg 6.10.3.233 out:</p> <p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFufSwbbB3ZEi0GIkIxYZYjLJ9wUT6y8LXlg3b6-4nOtI1G1QnuCw2etRxVkhM-OqLhnpBcnnVZHvf3i7w9ZhhRbRBIfsalFbOrHkUBsPYqhkG_EEGHiSj5HpG6ThqURyEj_iR/s1600-h/image%5B2%5D.png"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="125" alt="image" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6e3cds4goINGY4FUrdhoKz8Q0_3EpV_IPy9aPXiSWKS7kNw_F9rZ6YHGCL0_saBbcFiFWUwx6OpqOs7n4XjG7575eKG4X4B-SBmqB2cRSZueTPvnBlIfK0MfCsT1AH-aCPprV/?imgmax=800" width="244" border="0" /></a> </p> <p>Here's the direct download <a href="http://msdl.microsoft.com/download/symbols/debuggers/dbg_x86_6.10.3.233.msi">link</a>. Unfortunately there's still no inherent <a href="http://voneinem-windbg.blogspot.com/2007/04/windbg-6750-released.html">managed debugging support as we had with 6.7.5.0</a> so I'll keep this version (until the end of days?)...</p> <p> </p> <p><strong><font color="#ff0000">UPDATE</font></strong></p> <p><strong><font color="#ff0000">Pat Styles </font></strong><a href="http://groups.google.com/group/microsoft.public.windbg/browse_thread/thread/03530a0643372f66#"><strong><font color="#ff0000">explains</font></strong></a><strong><font color="#ff0000"> in the </font></strong><a href="http://groups.google.com/group/microsoft.public.windbg/topics"><strong><font color="#ff0000">windbg newsgroup</font></strong></a><strong><font color="#ff0000"> the background why this managed debugging feature made it's way in public and why it has been taken way very soon. <br />So sad it is: likely there  will be NO managed debugging support for windbg in the future. So be happy if you kept your 6.7.5.0 and make an extra backup!</font></strong></p> <p> </p> <p>As <a href="http://voneinem-windbg.blogspot.com/2007/10/windbg-version-6840-is-out-im-still.html">last time</a> the highlights are very roughly described so I pasting the full list of relnotes.txt here for googles convenience ;-)</p> <p>I marked the features I very personally like in <font color="#00ff00">green</font></p> <p>Important changes in WinDbg 6.10: </p> <p>  * CE6 dump processing now works. <br />  * NT4 no longer supported. <br />  * Enable installation of the USB debug driver on 64bit versions of Windows. <br />  * Install USBView.exe as part of the debugger package. <br />  * Enable use of any 1394 cards in target machines. <br />  * Improve 1394 debug driver reliability, responsiveness and performance. <br />  * Improve error messages for specific 1394 driver installation failures. <br />  * Prevent loss of 1394 debug connections after host sleep or hibernate. <br />  * Change the 1394 debug driver to write every sent packet to the target <br />    regardless of whether the previously sent packet was acknowleged. <br />  * Ensure the installed 1394 debug driver is new enough to work properly with <br />    the current release of the debugger.  (Update the 1394 driver to the <br />    version shipped in this debugger package.) <br />  * Automatically reconnect to broken in target machines when doing 1394 <br />    kernel debugging. <br /><font color="#00ff00">  * Introduce new ".dump /mA ..." option to generate user-mode minidump. <br />    The new behaiour is the same as ".dump /ma" except that it will ignore <br />    inacessable memory read failure and continue minidump generation, <br />    instead of bailing out immediately. </font><font color="#0000ff">>>I've seen this behavior quite often - so thank's for that!<<</font> <br />  * Add CHK/FRE information in user-mode minidump. Only dumps generated from <br />    the new debugger will carry this information. <br />  * Include breakpoint exception in user-mode minidump when the dump is <br />    generated after a breakpoint hits. This enables debugger to show the <br />    current process/thread information when debugging a dump file. <br />  * Fix debugger hang when writing lsass dump files. <br />  * Fix debugging of WOW64 minidumps containing managed code. <br />  * Force the WOW64 extension to convert contexts as appropriate. <br />  * Don't sign extend addresses of 32 bit user mode processes in the 64 bit <br />    debugger. <br />  * Set 64bit debugger as the postmortem debugger for WOW64 applications. <br />  * Disable adding user-mode breakpoints in kd when KdIgnoreUmException is set. <br />  * Fix escape character ("\") parsing problem in the bl command. <br />  * Force bu breakpoint removal for unloaded modules. <br />  * Add automatic IPv6 detection to the "-remote tcp:..." remote protocol. <br />  * Fix .sleep to work when -ddefer is used. <br />  * Fix .tlist command <br />  * Performance improvements to windbg and other debuggers. <br />  * Notify users when workspace corruption is detected. <br />  * Fix the infinite recursion with corrupt workspaces in windbg. <br />  * Report the correct processor family values in !cpuid for all processors. <br />  * Attempt to breakin only once per ctrl-c, ctrl-break instead of spinning <br />    in a loop on packet write failure. <br />  * Print the target computername in the debugger vertarget output when symbol <br />    information is available. <br /><font color="#00ff00">  * Tell debugger users how to break out of symbol loading, if it does not <br />    complete within a reasonable amount of time. <br /></font>  * Display source server spew state when .srcnoisy is run. Also display the <br />    source server only output filter state. <br />  * Workaround for Vista/DWM unpainted captions. <br />  * Add host debugger time when remote kernel target boots and shuts down. <br />  * Symstore: don't truncate long file names when compressing files. <br />  * Symchk: Add recursive symbol search option ("/sr") to force SymChk to <br />    recursively search (non-symbol server) symbol paths for matching symbols. <br />  * Add support in symsrv for getting file pointers from an http store and <br />    using them as is without caching them to the default downstream store. <br />  * Agestore tool should not run on computers that do not update the <br />    last-access-time on files. <br /><font color="#00ff00">  * Many !analyze improvements. <br />  * Extensive debugger documentation updates.</font></p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com3tag:blogger.com,1999:blog-18989586.post-84937196582148179322008-10-07T09:05:00.001+02:002008-10-07T09:05:20.740+02:00How to modify memory of a running process<p>Kahled <a href="http://voneinem-windbg.blogspot.com/2007/06/scan-full-process-memory-for-pattern.html?showComment=1223334000000#c246344058147996839">asked</a> me how to modify memory at a certain addess.</p> <p>So here we go...</p> <p>Launch notepad and type "hallo"</p> <p><a href="http://lh4.ggpht.com/VolkerVonEinem/SOsKKhSGEOI/AAAAAAAAAFc/hIo7OpUkGvE/s1600-h/image%5B2%5D.png"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="127" alt="image" src="http://lh5.ggpht.com/VolkerVonEinem/SOsKLBuVS-I/AAAAAAAAAFg/W_-nklopE9U/image_thumb.png?imgmax=800" width="244" border="0" /></a> </p> <p>Now attach windbg to it and search for the "hallo" string using the <a href="http://voneinem-windbg.blogspot.com/2007/06/scan-full-process-memory-for-pattern.html">scan memory pattern</a>:</p> <p>0:001> s -u 0x00000000 L?0xffffffff "hallo"</p> <p><font color="#ff0000">00184958</font>  0068 0061 006c 006c 006f 0000 0000 0000  h.a.l.l.o.......</p> <p>You can now modify the string at memory location <font color="#ff0000">00184958</font> with the <a href="http://msdn.microsoft.com/en-us/library/cc266701.aspx">e (Enter Values) command</a>:</p> <p>0:001> eu <font color="#ff0000">00184958</font> "hello"</p> <p>You will see after letting the process run again (g) that the german "hallo" switched to an english "hello":</p> <p><a href="http://lh5.ggpht.com/VolkerVonEinem/SOsKLsHjfuI/AAAAAAAAAFk/7rJb0jJaxKY/s1600-h/image%5B12%5D.png"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="152" alt="image" src="http://lh6.ggpht.com/VolkerVonEinem/SOsKMNN2yRI/AAAAAAAAAFo/0uf7A5MN38Y/image_thumb%5B8%5D.png?imgmax=800" width="632" border="0" /></a></p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com2tag:blogger.com,1999:blog-18989586.post-23328449750072473472008-09-17T10:44:00.001+02:002008-09-17T15:51:00.690+02:00Amazing helper .cmdtree<p>In <a href="http://blogs.msdn.com/debuggingtoolbox/about.aspx">Roberto Farah's</a> most recent <a href="http://blogs.msdn.com/debuggingtoolbox/archive/2008/09/17/special-command-execute-commands-from-a-customized-user-interface-with-cmdtree.aspx">post</a> he mentions the command .cmdtree which is available since version 6.6.7.5. <br />The result looks amazing and makes debugging with windbg a lot more productive and it makes it much more useful for especially for newbies:</p> <p><a href="http://lh4.ggpht.com/VolkerVonEinem/SNDDUryVr4I/AAAAAAAAAFU/GweP5C3QQcI/s1600-h/image%5B22%5D.png"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="688" alt="image" src="http://lh6.ggpht.com/VolkerVonEinem/SNDDWLtBwpI/AAAAAAAAAFY/HJFgnmpVHIw/image_thumb%5B20%5D.png?imgmax=800" width="684" border="0" /></a> </p> <p>I've put the <a href="http://vve.gmxhome.de/docs/CMDTREE.TXT">text file</a> proposed by Roberto on my public share for your (and my) convenience and added some stuff, that I found to be useful... <br />Currently I don't know how to put comments into this file format. <br />If anyone can tell me I would like to add a tribute notice into the text.</p> <p>PS: Unfortunately I was not able to set the command .cmdtree CMDTREE.TXT as initial command via -c switch in  [HKEY_CLASSES_ROOT\WinDbg.DumpFile.1\shell\Open\command]. For some reasons windbg doesn't find the file via startup command. Any advice on how to achieve this is welcome.</p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com5tag:blogger.com,1999:blog-18989586.post-34004086078864326042008-09-11T09:29:00.001+02:002008-09-11T09:29:22.066+02:00Interview with Mario Hewardt and Daniel Pravat on Channel9<p>The authors of the excellent <a href="http://advancedwindowsdebugging.com/">book Advanced Windows Debugging</a> <a href="http://advancedwindowsdebugging.com/portal/portal_m_blog.htm">Mario Hewardt</a> and <a href="http://www.advancedwindowsdebugging.com/forums/blogs/daniel/default.aspx">Daniel Pravat</a> are <a href="http://channel9.msdn.com/posts/Charles/Advanced-Windows-Debugging-An-Introduction/">talking</a> about debugging windows and the drivers to write the book on <a href="http://channel9.msdn.com">Channel9</a>.</p> <p>The book is great and should be in the book shelf of every windows developer, because it describes a structured way of doing common windows debugging tasks. As the authors state, an important reason for writing this book was the fact, that there was no such resource available.</p> <p>There's a lot talking about debugging threading issues which is sometimes really hard stuff. Looking at the fact that modern computers have more and more cores, writing, understanding and debugging multithreaded code will become more and more important.</p> <p>One might argue that in times of managed code unmanaged debugging is out of date. Mario and Daniel explain in this interview that understanding the details behind the scenes is essential for doing efficient debugging. You have to understand the basics on how the CLR works and the CLR is written in unmanaged code.</p> <p>(If you interesting in more debugging of managed code I strongly recommend <a href="http://www.wintellect.com/BookInformation.aspx?ASIN=0735622027">Debugging Microsoft .NET 2.0 Applications</a> by <a href="http://www.wintellect.com/CS/blogs/jrobbins/default.aspx">John Robbins</a>)</p> <p>At the end of the interview they do a little live debugging on a program producing resource leaks... nice. <br />(I've been looking into code analysis tools in the last couple of weeks and I bet the resource leak would have been found by some of those tools without running the program a single time. But it's just for demo and the best code analysis tools will leave enough bugs in your code that will push you in front of the debugger)</p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-58654711322452865872008-05-30T15:00:00.003+02:002009-03-30T12:30:45.921+02:00Debugging reference count leaks<p>One of the hardest things is debugging a reference count leak. <a href="http://msdn.microsoft.com/en-us/library/ms680573%28VS.85%29.aspx">COM</a> objects lifetime depends on the reference count (read <a href="http://msdn.microsoft.com/en-us/library/ms687260%28VS.85%29.aspx">here</a> for more...). So each client of a COM object must call <a href="http://msdn.microsoft.com/en-us/library/ms691379.aspx">AddRef</a> on the <a href="http://msdn.microsoft.com/en-us/library/ms680509%28VS.85%29.aspx">IUnknown</a> interface when going to use it and it must call <a href="http://msdn.microsoft.com/en-us/library/ms682317%28VS.85%29.aspx">Release</a> when done. If any client (and there might be many many of a single one) violates this rule you get into severe trouble. </p> <h3>Scenarios</h3> <h4>1.) Number of Release calls = Number of AddRef calls</h4> <p>This is the normal scenario: As soon as no client needs the server object anymore it is getting destroyed</p> <h4>2.) Number of Release calls > Number of AddRef calls</h4> <p>If Release is called one time too often another client might crash because the server get's destroyed too early - bad thing here is that you see the crash in some place but this does not tell you where is root cause is located. All you know is which objects reference count has been corrupted.</p> <h4>3.) Number of AddRef calls > Number of Release calls</h4> <p>If AddRef is called one time too often the reference count never reaches 0 and hence the server object never get's destroyed. This is causing memory leaks and also might cause resource leaks. The effect of this scenario is much less obvious: You might see memory increasing over time and/or performance degrade and/or resources to be locked when they should be unlocked again.</p> <p> </p> <p>Finding the place where the unbalanced AddRef/Release occurred might be like finding the needle in the hay. I did research in the Google reachable web but didn't find a good tool available that really assist's in this task. Luckily <a href="http://blogs.msdn.com/saraford/articles/454276.aspx">Sara Ford</a> described in this <a href="http://blogs.msdn.com/saraford/archive/2005/05/03/414385.aspx">post</a> the first step you need to take in order to get the data necessary to drill down into the problem.</p> <p>Somehow I didn't manage to set the trace points in Visual Studio 2005 (can anybody tell me how to set a break point on a single objects AddRef, Release methods?) so I launched my beloved WinDbg.</p> <p>First I created script to create me an xml snippet for an event that alters the ref count (I didn't find a better name so I called it ToXml.txt and placed it into my scripts folder):</p> <p>.printf "-->\n<Event><Ref>%d</Ref><![CDATA[",poi(${$arg1}) <br />k100 <br />.printf "]]></Event>\n<!--\n"</p> <p>Then I placed a break point on the server objects constructor</p> <p>bp MyServer!CMyClass::CMyClass</p> <p>When the breakpoint hit, I stepped out <Shift>+<H11> into CComCreator::CreateInstance and then stepped over the p->SetVoid(pv); call in this class. <br />(I think it should be possible to set a breakpoint directly at MyServer!ATL::CComCreator<ATL::CComObject<CMyClass> >::CreateInstance+0xb1, but I didn't try...)</p> <p>Now I gathered the address of m_dwRef by:</p> <p>0:000> ?? &(p->m_dwRef) <br />long * <span style="color: rgb(51, 204, 0);">0x110d724c</span></p> <p>Next thing to do is setting the data breakpoint by:</p> <p>ba w4 <span style="color: rgb(51, 204, 0);">0x110d724c </span>"$$>a<C:/windbg/scripts/ToXml.txt 0f084cb4;gc"</p> <p>(you might need to change the path 'C:/windbg/scripts/')</p> <p>With <a href="http://msdn.microsoft.com/en-us/library/cc266851.aspx">.logopen</a> we make sure that we directly write all events into an logfile:</p> <p>.logopen c:\temp\Events.xml</p> <p>Now let the application run with 'g' or <F5> and do whatever creates your ref counting problem.</p> <p>When done break into and close the log with <a href="http://msdn.microsoft.com/en-us/library/cc266834.aspx">.logclose</a>.</p> <p>At this point we are half the way through. The Events.xml we created is not valid xml. You need to add </p> <p><?xml version="1.0" encoding="UTF-8"?> <br /><Events> <br /><!--</p> <p>at the beginning and</p> <p>--></Events></p> <p>at the end.</p> <p> </p> <p>Now comes the tooling. In my scenario I had around 1400 Events - a little tedious to analyze all by hand.</p> <p>So I created "Volkers RefCount Buster" which does the following:</p> <p>1.) After loading the file (enter path in first text box and press Start) all events are identified for either beeing AddRef or Release</p> <p>2.) Then the call stack is taken to group the events:</p> <p>First action is to exclude events that match the pattern entered in the second text box (exclude pattern):</p> <!-- {\rtf1\ansi\ansicpg\lang1024\noproof65001\uc1 \deff0{\fonttbl{\f0\fnil\fcharset0\fprq1 Courier New;}}{\colortbl;??\red0\green0\blue255;\red255\green255\blue255;\red0\green0\blue0;\red43\green145\blue175;}??\fs24 \cf1 var\cf0 includeQuery = \cf1 from\cf0 frame \cf1 in\cf0 \cf1 this\cf0 .StackFrames.Frames\par ?? \cf1 where\cf0 \cf4 String\cf0 .IsNullOrEmpty(ExcludePattern) ? \cf1 true\cf0 : !excludePattern.IsMatch(frame)\par ?? \cf1 select\cf0 frame;} --> <div style="background: white none repeat scroll 0% 0%; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;font-family:courier new;font-size:12pt;color:black;"> <p style="margin: 0px;"><span style="color:blue;">var</span> includeQuery = <span style="color:blue;">from</span> frame <span style="color:blue;">in</span> <span style="color:blue;">this</span>.StackFrames.Frames</p> <p style="margin: 0px;"> <span style="color:blue;">where</span> <span style="color: rgb(43, 145, 175);">String</span>.IsNullOrEmpty(ExcludePattern) ? <span style="color:blue;">true</span> : !excludePattern.IsMatch(frame)</p> <p style="margin: 0px;"> <span style="color:blue;">select</span> frame;</p> </div> <p> Then the remaining frames are searched for the selection pattern:</p> <div style="background: white none repeat scroll 0% 0%; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;font-family:courier new;font-size:12pt;color:black;"> <p style="margin: 0px;"><span style="color:blue;">var</span> selectionQuery = <span style="color:blue;">from</span> frame <span style="color:blue;">in</span> includeQuery</p> <p style="margin: 0px;"> <span style="color:blue;">let</span> match = selectionPattern.Match(frame)</p> <p style="margin: 0px;"> <span style="color:blue;">where</span> match.Success</p> <p style="margin: 0px;"> <span style="color:blue;">select</span> match.Value;</p> </div> <p>and the top most match is taken:</p> <p> <span style="color:blue;">string</span> sourceGroup = selectionQuery.FirstOrDefault();</p> <p>the all events are grouped into the found source groups:</p> <p> </p> <div style="background: white none repeat scroll 0% 0%; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;font-family:courier new;font-size:12pt;color:black;"> <p style="margin: 0px;"><span style="color:blue;">var</span> ResultQuery = <span style="color:blue;">from</span> refCountEvent <span style="color:blue;">in</span> refCountEvents</p> <p style="margin: 0px;"> <span style="color:blue;">group</span> refCountEvent <span style="color:blue;">by</span> refCountEvent.SourceGroup <span style="color:blue;">into</span> g</p> <p style="margin: 0px;"> <span style="color:blue;">select</span> g;</p> <p style="margin: 0px;"> </p> <p style="margin: 0px;"><span style="font-family:Trebuchet MS;"><span style="font-size:85%;">Then the number of AddRefs and Releases is calculated for each group and accumulated:</span></span></p> <div style="background: white none repeat scroll 0% 0%; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;font-family:courier new;font-size:12pt;color:black;"> </div> </div> <div style="background: white none repeat scroll 0% 0%; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;font-family:courier new;font-size:12pt;color:black;"><span style=";font-family:Trebuchet MS;font-size:85%;" ></span></div> <div face="courier new" size="12pt" color="black" style="background: white none repeat scroll 0% 0%; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;"> <p style="margin: 0px;"><span style="color:blue;">foreach</span> (<span style="color:blue;">var</span> ResultSet <span style="color:blue;">in</span> ResultQuery)</p> <p style="margin: 0px;">{</p> <p style="margin: 0px;"> <span style="color:blue;">long</span> numOfAddRefs = (<span style="color:blue;">from</span> rce <span style="color:blue;">in</span> ResultSet.AsEnumerable()</p> <p style="margin: 0px;"> <span style="color:blue;">where</span> rce.RefCountType == <span style="color: rgb(43, 145, 175);">EventType</span>.AddRef</p> <p style="margin: 0px;"> <span style="color:blue;">select</span> rce).Count();</p> <p style="margin: 0px;"> </p> <p style="margin: 0px;"> <span style="color:blue;">long</span> numOfReleases = (<span style="color:blue;">from</span> rce <span style="color:blue;">in</span> ResultSet.AsEnumerable()</p> <p style="margin: 0px;"> <span style="color:blue;">where</span> rce.RefCountType == <span style="color: rgb(43, 145, 175);">EventType</span>.Release</p> <p style="margin: 0px;"> <span style="color:blue;">select</span> rce).Count();</p> <p style="margin: 0px;"> </p> <p style="margin: 0px;"> <span style="color:blue;">long</span> balance = numOfAddRefs - numOfReleases;</p> <p style="margin: 0px;">...</p> <p style="margin: 0px;"> </p> <p style="margin: 0px;">Now it's up to you to find the Exclude Pattern and Selection Pattern that will directly point you to the component or file, that is the culprit. Then you just need to look at those stacks that belong to the found bad guy and you will also be able to see the source line that created the problem.</p> <p style="margin: 0px;"> </p> </div> <p><a href="http://lh5.ggpht.com/VolkerVonEinem/SD_6aK-RJVI/AAAAAAAAAEU/PUqV6wkJR58/VolkersRefCountBuster%5B13%5D.png"><img style="border-width: 0px;" alt="VolkersRefCountBuster" src="http://lh5.ggpht.com/VolkerVonEinem/SD_6bK-RJWI/AAAAAAAAAEc/LGhZpiOjWjE/VolkersRefCountBuster_thumb%5B9%5D.png" width="677" border="0" height="417" /></a> </p> <p> You can download the sources and binaries <a href="http://www.vve.gmxhome.de/docs/RefCountBuster.zip">here...</a></p> <p> </p> <p>Have fun,</p> <p>Volker</p> <p> </p> <!-- {\rtf1\ansi\ansicpg\lang1024\noproof65001\uc1 \deff0{\fonttbl{\f0\fnil\fcharset0\fprq1 Courier New;}}{\colortbl;??\red0\green0\blue255;\red255\green255\blue255;\red0\green0\blue0;}??\fs24 \cf1 var\cf0 selectionQuery = \cf1 from\cf0 frame \cf1 in\cf0 includeQuery\par ?? \cf1 let\cf0 match = selectionPattern.Match(frame)\par ?? \cf1 where\cf0 match.Success\par ?? \cf1 select\cf0 match.Value;} --> <div style="background: white none repeat scroll 0% 0%; font-size: 12pt; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial; color: black; font-family: courier new;"> <p style="margin: 0px;"></p> </div>Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com7tag:blogger.com,1999:blog-18989586.post-30879839980493955132008-05-08T08:49:00.001+02:002008-05-08T08:49:26.920+02:00Windbg help is online...<p>Please correct me if I'm wrong, but at least some month ago there was online help for Debugging Tools for Windows on MSDN or anywhere else.</p> <p>Now Microsoft put the (very good and helpful!) help <a href="http://msdn.microsoft.com/en-us/library/cc267445.aspx">online</a>:</p> <p><a href="http://msdn.microsoft.com/en-us/library/cc267445.aspx"><img style="border-right: 0px; border-top: 0px; border-left: 0px; border-bottom: 0px" height="435" alt="image" src="http://lh4.ggpht.com/VolkerVonEinem/SCKidSK0egI/AAAAAAAAAEM/_0c5ueRusFU/image%5B11%5D.png" width="679" border="0" /></a> </p> <p>Have fun...</p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-54694877413894411842008-05-06T09:45:00.002+02:002008-07-25T15:55:31.908+02:00Windbg version 6.9.3.113 is out (I'm still keeping 6.7.5.0)<p>Microsoft <a href="http://www.wintellect.com/cs/blogs/jrobbins/archive/2008/05/05/windbg-6-9-3-113-released.aspx">released</a> a new version of Debugging Tools for windows. As usual the installation bits can be grabbed <a href="http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx#a">here..</a></p> <p>Unfortunately the <a href="http://voneinem-windbg.blogspot.com/2007/04/windbg-6750-released.html">integrated managed debugging</a> that accidentally came in with version 6.7.5.0 is still missing :-(</p> <p>Anyway here's the <a href="http://www.microsoft.com/whdc/devtools/debugging/whatsnew.mspx">list of changes</a>:</p> <h4>Highlights in Version 6.9.3.113</h4> <p>In this release, you will find better performance on systems with greater memory as well as those with large CPU counts. Better performance and reliability on transport initialization. Enhancements to <b>dt, sx, z, !defwrites, !sysinfo, !gflags, Symsrv,</b> as well as several others. For further details, please read the RELNOTES.TXT provided in the package. </p> <p>Taken from the installation relnotes.txt:</p> <p>* Fix kd to function properly when debugging 256 processor machine. <br />* Fix windbg window dragging performance problems when running under AERO <br /> on Vista. <br />* Alert the user when a debug transport is already opened by another <br /> instance of the debugger. <br />* Only attempt driver install after opening the transport fails with file <br /> not found. <br />* Add /LARGEADDRESSAWARE to debugger executables (cdb/kd/ntsd/windbg). <br />* Update vmdemux tool <br />* Fix pdbcopy.exe tool <br />* "dt" would display enumerant symbolic names for enumeration-typed bit- <br /> field members. <br />* Make "dt" member field match case-insensitive. <br />* Support wildcard module name in "dt" command. For example, "dt <br /> adv!*RegQuery*". <br />* "dt/dv" would output more information indicating that this is an empty <br /> string (the default display "") or this is memory read failure (new output <br /> "--- memory read error at address ..."). <br />* "sx? ud" commands can now use an image name (for example, ntdll.dll) as <br /> well as a module name (ntdll). <br />* Fix ".dbgdbg" command failure when debugger is installed in a directory <br /> that contains spaces (for example, "c:\Program Files\Debuggers"). <br />* Fix "z" command loop counter reset problem. <br />* Debugger extension would be loaded using LOAD_WITH_ALTERED_SEARCH_PATH <br /> so that dependent binaries could be loaded from the same directory where <br /> loaded extension resides (and the directory is not part of search path). <br />* "!defwrites" (in kdexts.dll) will not query nt!MmThrottleTop and <br /> nt!MmThrottleBottom values in Windows Vista. <br />* Fix for "!sysinfo cpuinfo". <br />* Fix for "!sysinfo gbl" infinite loop problem. <br />* Fix for DisplayFlags() has output string buffer overrun when using <br /> "!handle" (in ntsdexts.dll). <br />* Fix for "!gflags" command. <br />* Fix so that "fltkd" and "boot" debugger extensions run on pre-Win7 OS. <br />* Fix Symstore/SymChk improved detection for resource-only binaries <br />* Continued on-going improvements to !analyze </p> <p> </p> <p>Good to know, that you can have multiple versions of windbg on the same machine by simply x-copying it - hope that someday there's no need to write: "I'm still keeping 6.7.5.0"</p>Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com1tag:blogger.com,1999:blog-18989586.post-44676853407231663662008-03-06T15:54:00.001+01:002008-03-06T15:54:22.053+01:00Slow symbol loads because of unresolved breakpoints<p>I was wondering why debugging one specific application took much more time than it used before. <br />Additionally I always got this suspicious message:</p> <p>>> <br />Integrated managed debugging does not support enumeration of symbols. <br />See http://dbg/managed.htm for more details. <br /><<</p> <p>The root cause of the slow down was a unresolved breakpoint that caused searching for a match after each module load.</p> <p><font face="Arial" size="2"> <br />0:012> bl <br /> 0 eu             0001 (0001) (v)</font></p> <p>(the u in eu stands for unresolved...) <br />It is a common illness I'm suffering from to miss the Ctrl button when doing cut and paste. The result is a breakpoint on 'v' :-(</p> <p>What helped was a simple 'bc*' to clear all breakpoints and I was able to debug at the speed I was used to...</p> Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com1tag:blogger.com,1999:blog-18989586.post-80910031260023096462008-02-21T16:57:00.008+01:002008-02-22T08:04:46.380+01:00Shooting the PAGE_GUARD flag with MiniDumpWithIndirectlyReferencedMemoryA colleague of mine (thanks Ralf for pointing this out!) told me that using <a href="http://msdn2.microsoft.com/en-us/library/ms680519.aspx">MiniDumpWithIndirectlyReferencedMemory</a> in <a href="http://msdn2.microsoft.com/en-us/library/ms680360%28VS.85%29.aspx">MiniDumpWriteDump</a> can cause a nasty crashes.<br />Following the "in 99.9% of the cases it is your own fault" pattern I suspected the problem to be somewhere else but in dbghelp. Ralf kindly provided me with a sample project which I condensed a bit to fit on a single page:<br /><br /><pre><span class="comment">// GuardPageDump.cpp : Defines the entry point for the console application.<br />//<br /></span><span class="pre">#include "stdafx.h"<br />#include "C:/Program Files/Debugging Tools for Windows/sdk/inc/dbghelp.h"<br />#pragma comment( lib, "C:/Program Files/Debugging Tools for Windows/sdk/lib/i386/dbghelp.lib")<br /><br />#include "process.h"<br /></span><span class="type"><br />void</span> BigFunc<span class="operator"> ()<br />{</span><span class="type"><br />char</span> sBigBuffer<span class="operator"> [</span><span class="int">20000</span><span class="operator">] = {</span><span class="char">'\0'</span><span class="operator">};</span><br />printf<span class="operator"> (</span><span class="string">"BigFunc was called!\nsBigBuffer: %s\n"</span><span class="operator">,</span>sBigBuffer<span class="operator">);<br />}</span><span class="type"><br /><br />void</span> ProblemFunc<span class="operator">(</span>HANDLE hWaitForMe<span class="operator">)<br />{</span><span class="type"><br />char</span> sDummy1<span class="operator">[] = {</span><span class="char">'A'</span><span class="operator">,</span><span class="char">'\0'</span><span class="operator">};</span><span class="type"><br /><br />unsigned long</span> iDummy<span class="operator"> =</span><span class="keyword"> reinterpret_cast</span><span class="operator"><</span><span class="type">unsigned long</span><span class="operator">>(&(</span>sDummy1<span class="operator">[</span><span class="int">0</span><span class="operator">]));</span><span class="comment"><br /><br />// let iDummp seem like a pointer pointing to the guarded page<br /></span> iDummy<span class="operator"> -=</span><span class="int"> 0x2000</span><span class="operator">;</span><br />printf<span class="operator"> (</span><span class="string">"Integer value: %d\n"</span><span class="operator">,</span> iDummy<span class="operator">);</span><span class="comment"><br /><br />//make sure the integer value pointing to the guard page area is on the stack, when MiniDumpWriteDump is called from another stack<br />//during wait we will issue a dump creation on the main thread<br /></span> WaitForSingleObject<span class="operator">(</span>hWaitForMe<span class="operator">,</span>INFINITE<span class="operator">);</span><br /><br />printf<span class="operator"> (</span><span class="string">"Calling BigFunc crashes since stack can no longer be extended\n"</span><span class="operator">);</span><br /><br />BigFunc<span class="operator">();</span><br /><br />printf<span class="operator"> (</span><span class="string">"Integer value: %d\n"</span><span class="operator">,</span> iDummy<span class="operator">);<br />}</span><span class="type"><br /><br />void</span> ProblemFuncThread<span class="operator">(</span><span class="type">void</span><span class="operator"> *</span> p<span class="operator">)<br />{</span><br />ProblemFunc<span class="operator">(</span><span class="keyword">reinterpret_cast</span><span class="operator"><</span>HANDLE<span class="operator">>(</span>p<span class="operator">));</span><br />_endthread<span class="operator">();<br />}</span><span class="type"><br /><br />int</span> _tmain<span class="operator">(</span><span class="type">int</span> argc<span class="operator">,</span> _TCHAR<span class="operator">*</span> argv<span class="operator">[])<br />{</span><br />printf<span class="operator"> (</span><span class="string">"This program demonstrates the damaging effect of creating a userdump with MiniDumpWithIndirectlyReferencedMemory\n"</span><span class="operator">);</span><br /><span class="operator"></span><br />HANDLE hWaitForMe<span class="operator"> =</span> CreateEvent<span class="operator">(</span>NULL<span class="operator">,</span>FALSE<span class="operator">,</span>FALSE<span class="operator">,</span>NULL<span class="operator">);</span><br /><br />printf<span class="operator"> (</span><span class="string">"Calling ProblemFunc on a different thread\n"</span><span class="operator">);<br /></span> uintptr_t hThread<span class="operator"> =</span> _beginthread<span class="operator">(</span>ProblemFuncThread<span class="operator">,</span><span class="int">0</span><span class="operator">,</span>hWaitForMe<span class="operator">);</span><span class="comment"><br />// give the thread time to start<br /></span><span class="operator"> ::</span>Sleep<span class="operator">(</span><span class="int">1000</span><span class="operator">);</span><br /><br />printf<span class="operator"> (</span><span class="string">"Creating a userdump of type MiniDumpWithIndirectlyReferencedMemory\n"</span><span class="operator">);</span><br />printf<span class="operator"> (</span><span class="string">"Resets guard page flag on the page pointed to by iDummy\n"</span><span class="operator">);</span><br />HANDLE hFile<span class="operator"> =</span> CreateFile<span class="operator">(</span>_T<span class="operator">(</span><span class="string">"c:\\temp\\test_indirect.dmp"</span><span class="operator">),</span> GENERIC_READ<span class="operator"> |</span> GENERIC_WRITE<span class="operator">,</span><span class="int"> 0</span><span class="operator">,</span> NULL<span class="operator">,</span> CREATE_ALWAYS<span class="operator">,</span> FILE_ATTRIBUTE_NORMAL<span class="operator">,</span> NULL<span class="operator"> );</span><br />MiniDumpWriteDump<span class="operator">(</span>GetCurrentProcess<span class="operator">(),</span>GetCurrentProcessId<span class="operator">(),</span>hFile<span class="operator">,(</span>MINIDUMP_TYPE<span class="operator">) (</span>MiniDumpWithIndirectlyReferencedMemory<span class="operator">),</span><span class="int">0</span><span class="operator">,</span><span class="int">0</span><span class="operator">,</span><span class="int">0</span><span class="operator">);</span><br /><br />CloseHandle<span class="operator">(</span>hFile<span class="operator">);</span><br /><br />SetEvent<span class="operator">(</span>hWaitForMe<span class="operator">);</span><br />CloseHandle<span class="operator">(</span>hWaitForMe<span class="operator">);</span><br /><br />WaitForSingleObject<span class="operator">(</span><span class="keyword">reinterpret_cast</span><span class="operator"><</span>HANDLE<span class="operator">>(</span>hThread<span class="operator">),</span>INFINITE<span class="operator">);</span><span class="flow"><br />return</span><span class="int"> 0</span><span class="operator">;<br />}</span><br /></pre><br />This sample assumes you have installed Debugging Tools for Windows to the default location (C:/Program Files/Debugging Tools for Windows) and you have selected to install the SDK as well.<br /><br />I'm setting two breakpoints: One on the line that is calling MiniDumpWriteDump - another on the line before calling BicFunc.<br /><br />First let's have a look at the state before calling MiniDumpWriteDump:<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrm6HhxrgwUC8OY-XMppmL6lC1hLzE2b803J13bFcwwtmIwdWkaGQBOEuPQmHRuWxF1DhgLPtnKXjKOj2XJJpmTSdYlfpRXMKVGt4Tlqq9P1E3IzZ-94Uj2fArWFwxtJMzTgVi/s1600-h/GuardPageDump1.png"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrm6HhxrgwUC8OY-XMppmL6lC1hLzE2b803J13bFcwwtmIwdWkaGQBOEuPQmHRuWxF1DhgLPtnKXjKOj2XJJpmTSdYlfpRXMKVGt4Tlqq9P1E3IzZ-94Uj2fArWFwxtJMzTgVi/s320/GuardPageDump1.png" alt="" id="BLOGGER_PHOTO_ID_5169496516532509522" border="0" /></a><br /><br />Switching to thread 001 we will notice the value of iDummy = 0x88df38 on the stack that is waiting:<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGE5eoUDphI0z0iQnlmSYDH4l3uLWZ2_SrkitEQViQteI-M5jAxjL0Y1m2acO2qxGFaD9NEG8eOC_PhOZpPjDQtaEroW0BbfUGLEzujlZgqTi13i1J36QANeVjpAWmkmTsgG_H/s1600-h/GuardPageDump2.png"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGE5eoUDphI0z0iQnlmSYDH4l3uLWZ2_SrkitEQViQteI-M5jAxjL0Y1m2acO2qxGFaD9NEG8eOC_PhOZpPjDQtaEroW0BbfUGLEzujlZgqTi13i1J36QANeVjpAWmkmTsgG_H/s320/GuardPageDump2.png" alt="" id="BLOGGER_PHOTO_ID_5169498268879166306" border="0" /></a><br /><br />Now before calling MiniDumpWriteDump let's have a look at the memory layout:<br /><br /><span style=";font-family:courier new;font-size:85%;" >0:001> !vadump<br /><br />[...]<br /><br />BaseAddress: <span style="color: rgb(255, 0, 0);">00125000</span><br />RegionSize: <span style="color: rgb(255, 0, 0);">00001000</span><br />State: 00001000 MEM_COMMIT<br />Protect: 00000104 PAGE_READWRITE + PAGE_GUARD<br />Type: 00020000 MEM_PRIVATE<br /><br />[...]<br /><br />BaseAddress: <span style="color: rgb(255, 0, 0);">0088d000</span><br />RegionSize: <span style="color: rgb(255, 0, 0);">00001000</span><br />State: 00001000 MEM_COMMIT<br />Protect: 00000104 PAGE_READWRITE + PAGE_GUARD<br />Type: 00020000 MEM_PRIVATE</span><br /><br />Ok, there is a PAGE_GUARD flag for each thread...<br /><br /><span style="font-size:85%;"><span style="font-family:courier new;">0:000> ~0s;!teb</span><br /><span style="font-family:courier new;">[...]</span><br /><span style="font-family:courier new;">GuardPageDump!wmain+0x91:</span><br /><span style="font-family:courier new;">00401991 6a00 push 0</span><br /><span style="font-family:courier new;">TEB at 7ffdf000</span><br /><span style="font-family:courier new;">[...]</span><br /><span style="font-family:courier new;"> StackLimit: 00126000</span><br /><span style="font-family:courier new;">[...]</span><br /><span style="font-family:courier new;">0:000> ~1s;!teb</span><br /><span style="font-family:courier new;">[...]</span><br /><span style="font-family:courier new;">ntdll!KiFastSystemCallRet:</span><br /><span style="font-family:courier new;">771d9a94 c3 ret</span><br /><span style="font-family:courier new;">TEB at 7ffde000</span><br /><span style="font-family:courier new;">[...]</span><br /><span style="font-family:courier new;"> StackLimit: 0088e000</span><br /><span style="font-family:courier new;">[...]</span><br /></span><br />Adding the RegionSize to BaseAddress gives us the StackLimit observed by !teb.<br /><br />Now comes the clue: iDummy holds the value 0x88df38 (no pointer) that represents an address in the guarded page. Dbghelp does not know by looking at the stack if this is a pointer or value and follows the indirections. Looking at the memory layout after call to MiniDumpWriteDump reveals the problem:<br /><span style="font-size:85%;"><span style="font-family:courier new;">0:001> !vadump</span><br /><span style="font-family:courier new;">[...]</span><br /><span style="font-family:courier new;">BaseAddress: 00125000</span><br /><span style="font-family:courier new;">RegionSize: 00001000</span><br /><span style="font-family:courier new;">State: 00001000 MEM_COMMIT</span><br /><span style="font-family:courier new;">Protect: 00000104 PAGE_READWRITE + PAGE_GUARD</span><br /><span style="font-family:courier new;">Type: 00020000 MEM_PRIVATE</span><br /><br /><span style="font-family:courier new;">[...]</span><br /><br /><span style="font-family:courier new;">BaseAddress: 0088d000</span><br /><span style="font-family:courier new;">RegionSize: 00003000</span><br /><span style="font-family:courier new;">State: 00001000 MEM_COMMIT</span><br /><span style="color: rgb(255, 0, 0);font-family:courier new;" >Protect: 00000004 PAGE_READWRITE</span><br /><span style="font-family:courier new;">Type: 00020000 MEM_PRIVATE</span><br /></span><br />The PAGE_GUARD flag for thread 001 is gone! Now it's just a question of time until your application will crash without giving you any clue on the root cause of the problem:<br /><br /><span style=";font-family:courier new;font-size:85%;" >0:001> g<br />(13e0.ee4): Access violation - code c0000005 (first chance)<br />First chance exceptions are reported before any exception handling.<br />This exception may be expected and handled.<br />eax=0088c000 ebx=00bc2e30 ecx=0088b100 edx=771d9a94 esi=00000000 edi=00000000<br />eip=00401a07 esp=0088ff24 ebp=0088ff2c iopl=0 nv up ei pl nz na pe nc<br />cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010206<br />GuardPageDump!_chkstk+0x27:<br />00401a07 8500 test dword ptr [eax],eax ds:0023:0088c000=????????<br /></span><span style="font-size:85%;"><br /><span style="font-family:courier new;">0:001> g</span><br /><span style="font-weight: bold; color: rgb(255, 0, 0);font-family:courier new;" >(13e0.ee4): Access violation - code c0000005 (!!! second chance !!!)</span></span><br /><br />Next thing I will do is to tell Microsoft about this. Meanwhile I'll try to restore the PAGE_GUARD flag as a workaround fix using <a href="http://msdn2.microsoft.com/en-us/library/aa366898%28VS.85%29.aspx">VirtualProtext</a> but I'm not sure if this is really a good idea.<br /><br />BTW: I've tested this the latest version of dbghelp.dll I know:<br />A colleague of mine recently investigated a crash that could be deducted to the usage of MiniDumpWriteDump along with MiniDumpWithIndirectlyReferencedMemory flag. The problem is, that MiniDumpWriteDump clears the PAGE_GUARD flag if a value or pointer is on the a stack other than the stack that calls MiniDumpWriteDump.<br />As a result the process will crash later if the stack needs to be extended.<br />I've described the problem in detail here:<br />http://voneinem-windbg.blogspot.com/2008/02/shooting-pageguard-flag-with.html<br /><br />I would be nice to see fix for this, because the MiniDumpWithIndirectlyReferencedMemory is really useful.<br />I didn't investigate into the other flags of MINIDUMP_TYPE Enumeration but MiniDumpNormal and MiniDumpWithFullMemory seem to work without a problem.<br /><br />I've been testing this with the latest public version of dbghelp.dll<br /><span style="font-size:85%;"><span style="font-family:courier new;">0:001> lmvm dbghelp</span><br /><span style="font-family:courier new;">start end module name</span><br /><span style="font-family:courier new;">68af0000 68c05000 dbghelp (deferred) </span><br /><span style="font-family:courier new;"> Image path: C:\Users\voneinem\Documents\Visual Studio 2008\Projects\GuardPageDump\Release\dbghelp.dll</span><br /><span style="font-family:courier new;"> Image name: dbghelp.dll</span><br /><span style="font-family:courier new;"> Timestamp: Thu Sep 27 23:27:05 2007 (46FC2029)</span><br /><span style="font-family:courier new;"> CheckSum: 0010087A</span><br /><span style="font-family:courier new;"> ImageSize: 00115000</span><br /><span style="font-family:courier new;"> File version: 6.8.4.0</span><br /><span style="font-family:courier new;"> Product version: 6.8.4.0</span></span>Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com3tag:blogger.com,1999:blog-18989586.post-40334686999422443282008-02-13T10:42:00.007+01:002008-02-13T13:41:07.766+01:00Digging into Kernel SpaceThe company I am working for manufactures a laboratory device that is getting conntected via RS232 interface. As many laptop computers today do not have such an interface we must provide our customers with a good solution how to operate their instruments in such an environment. Therefore we identified a USB to RS232 adapter as a good solution and selected a cable with <a href="http://www.prolific.com.tw/eng/downloads.asp?ID=31">prolific</a> chipset <span style="width: 500px;"><span style="">PL-2303HX and drivers</span></span>.<br /><br />Lately I have seen sporadic cases of strange process hangs. The process that in writing into RS232 is stuck. I was able to attach windbg to that process and quickly realized that it was hanging in a call to <a href="http://msdn2.microsoft.com/en-us/library/aa365747%28VS.85%29.aspx">Kernel32!WriteFile</a>. As we are setting com timeouts via <a href="http://msdn2.microsoft.com/en-us/library/aa363437%28VS.85%29.aspx">kernel32!SetCommTimeouts</a> to 10 seconds in the worst case I would have expected the function to return in any case. This did not happen :-(<br /><br />Strange enough, killing the process was not possible anymore. No Task Manager, no Process Explorer and no kill -f was working. Finally I wasn't even able to shut down the system properly (XP SP2) and needed to kill it the hard way.<br /><br />OK I could not do anything more in user mode space that saying WriteFile is the bad guy. This is a bad explanation for customers. They want the system to work - regardless of how is the culprit.<br /><br />So I was entering new spaces - the kernel space...<br /><br />First task was to get a dump. <a href="http://www.osronline.com/index.cfm">OSR Online</a> offers a tool called <a href="http://www.osronline.com/article.cfm?article=153">Bang</a> which did the job.<br />Before I needed to configure the system to generate a full dump. Therefore right click on computer and select properties. Go to the advanced tab and click on Settings for Startup and recovery. In the 'Write debugging information' frame select 'Complete memory dump' and the dump file name:<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqwDK0ivDqCMDefEjGPr9qSYH4QFZvFb-vaQcqIcnroWpCtKGLWPFjjxy1qCooPCtKlMssjkt6rRrgKCctlPFVUt36V41IqzFonnt0c7u57RYw_BmwE3mv-IzMT8Q6nRcapdff/s1600-h/KernelDump.png"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqwDK0ivDqCMDefEjGPr9qSYH4QFZvFb-vaQcqIcnroWpCtKGLWPFjjxy1qCooPCtKlMssjkt6rRrgKCctlPFVUt36V41IqzFonnt0c7u57RYw_BmwE3mv-IzMT8Q6nRcapdff/s320/KernelDump.png" alt="" id="BLOGGER_PHOTO_ID_5166424270591105842" border="0" /></a><br /><br />Finish with OK and now start the Bang.exe (it seems to require a local admin account - so I used the runas...):<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglJARlFoIdduZxLtLyuygEvzw4gaIHcOljYww7OMTonjEnpo0hPxmjWphKJR4Q-2obQOdY5IXzSwWwDyd0BaHkgq2W1vS12KiLjm-Ecr1H_Mmb_NHrC2MJDaF-LucR8ALINW6j/s1600-h/Bang.png"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglJARlFoIdduZxLtLyuygEvzw4gaIHcOljYww7OMTonjEnpo0hPxmjWphKJR4Q-2obQOdY5IXzSwWwDyd0BaHkgq2W1vS12KiLjm-Ecr1H_Mmb_NHrC2MJDaF-LucR8ALINW6j/s320/Bang.png" alt="" id="BLOGGER_PHOTO_ID_5166426035822664514" border="0" /></a><br /><br />It is not hard to hit the nice red button and you'll see a nice blue screen ;-)<br /><br />After reboot the promised memory.dmp was available.<br /><br />But what next? I'm fully unexperienced in kernel debugging and I tried the 'poking around until you find something' pattern without success. Fortunately I have a MSDN subscription that offers two free support incidents. So I went to Bill and told him that I'm not happy with this. He said, please give me the dump and I'll see. Then he said it'll take songer than a day but not much. Finally took nearly tree weeks - never mind ;-)<br />I got the dump analysis out of Microsoft hands and this was pretty much pointing to the place I suspected (the USB to RS232 driver). Additionally he said that my process was accessing the same port from two different threads. This touched my pride as a developer and I needed to understand the magic bill was doing my dump. Finally I discovered that it was quite true as the two threads were accessing two different ports (I had in sum 4 ports on the system) - but that's forgiven.<br /><br />Now to the internals of the kernel dump analyis:<br /><br />I know the name of the hanging process. So first we need to get the processes:<br /><span style=";font-family:courier new;font-size:85%;" >0: kd> !process 0 0<br />[...]<br />PROCESS 820f4838 SessionId: 0 Cid: 01d4 Peb: 7ffd8000 ParentCid: 0608<br /> DirBase: 02a00440 ObjectTable: e4b36400 HandleCount: 209.<br /> Image: MyProcess.exe<br />[...]</span><br /><br />Now we need to get the details of the process:<br /><span style=";font-family:courier new;font-size:85%;" >0: kd> !process 820f4838 f<br />PROCESS 820f4838 SessionId: 0 Cid: 01d4 Peb: 7ffd8000 ParentCid: 0608<br /> DirBase: 02a00440 ObjectTable: e4b36400 HandleCount: 209.<br /> Image: </span><span style=";font-family:courier new;font-size:85%;" >MyProcess</span><span style=";font-family:courier new;font-size:85%;" >.exe<br /> VadRoot 81b9ffa8 Vads 288 Clone 0 Private 61253. Modified 84841. Locked 0.<br /> DeviceMap e4c3b450<br /> Token e4c02030<br /> ElapsedTime 1 Day 04:24:53.014<br /> UserTime 00:00:23.156<br /> KernelTime 00:00:20.000<br /> QuotaPoolUsage[PagedPool] 156580<br /> QuotaPoolUsage[NonPagedPool] 12612<br /> Working Set Sizes (now,min,max) (25572, 50, 345) (102288KB, 200KB, 1380KB)<br /> PeakWorkingSetSize 52956<br /> VirtualSize 372 Mb<br /> PeakVirtualSize 379 Mb<br /> PageFaultCount 203878<br /> MemoryPriority BACKGROUND<br /> BasePriority 8<br /> CommitCharge 63188<br /> DebugPort 81905360<br /><br /> THREAD 81a81020 Cid 01d4.0c20 Teb: 7ffaf000 Win32Thread: e397ceb0 WAIT: (DelayExecution) KernelMode Non-Alertable<br /> 81a81110 NotificationTimer<br /> <span style="font-weight: bold;">IRP List:</span><br /><span style="font-weight: bold;"> <span style="color: rgb(255, 0, 0);">816ecde0</span>: (0006,0220) Flags: 00000a30 Mdl: 00000000</span><br /> Not impersonating<br /> DeviceMap e4c3b450<br /> Owning Process 820f4838 Image: </span><span style=";font-family:courier new;font-size:85%;" >MyProcess</span><span style=";font-family:courier new;font-size:85%;" >.exe<br /> Wait Start TickCount 6562217 Ticks: 0<br /> Context Switch Count 408541 LargeStack<br /> UserTime 00:00:00.109<br /> KernelTime 00:00:00.421<br /> Win32 Start Address 0x77c3a341<br /> Start Address 0x7c810659<br /> Stack Init a9910000 Current a990fc04 Base a9910000 Limit a990d000 Call 0<br /> Priority 10 BasePriority 10 PriorityDecrement 0 DecrementCount 16<br /> ChildEBP RetAddr<br /> a990fc1c 80502e56 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])<br /> a990fc28 804faa13 nt!KiSwapThread+0x8a (FPO: [0,0,0])<br /> a990fc54 8057ca62 nt!KeDelayExecutionThread+0x1c9 (FPO: [Non-Fpo])<br /> a990fc78 8057e7ff nt!IopCancelAlertedRequest+0x52 (FPO: [Non-Fpo])<br /> a990fc94 8057c341 nt!IopSynchronousServiceTail+0xe1 (FPO: [Non-Fpo])<br /> <span style="font-weight: bold;">a990fd38 805409ac nt!NtWriteFile+0x5d7 (FPO: [Non-Fpo])</span><br /> a990fd38 7c90eb94 nt!KiFastCallEntry+0xfc (FPO: [0,0] TrapFrame @ a990fd64)<br />WARNING: Frame IP not in any known module. Following frames may be wrong.<br /> 0208fd70 00000000 0x7c90eb94<br /><br /> THREAD 8169da90 Cid 01d4.0ef0 Teb: 7ff9f000 Win32Thread: 00000000 WAIT: (DelayExecution) KernelMode Non-Alertable<br /> 8169db80 NotificationTimer<br /> <span style="font-weight: bold;">IRP List:</span><br /><span style="font-weight: bold;"> <span style="color: rgb(51, 204, 0);">816b3710</span>: (0006,0220) Flags: 00000a30 Mdl: 00000000</span><br /> Not impersonating<br /> DeviceMap e4c3b450<br /> Owning Process 820f4838 Image: </span><span style=";font-family:courier new;font-size:85%;" >MyProcess</span><span style=";font-family:courier new;font-size:85%;" >.exe<br /> Wait Start TickCount 6562217 Ticks: 0<br /> Context Switch Count 64786 <br /> UserTime 00:00:00.000<br /> KernelTime 00:00:00.000<br /> Win32 Start Address 0x77c3a341<br /> Start Address 0x7c810659<br /> Stack Init a9a5c000 Current a9a5bc04 Base a9a5c000 Limit a9a59000 Call 0<br /> Priority 6 BasePriority 6 PriorityDecrement 0 DecrementCount 0<br /> ChildEBP RetAddr<br /> a9a5bc1c 80502e56 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])<br /> a9a5bc28 804faa13 nt!KiSwapThread+0x8a (FPO: [0,0,0])<br /> a9a5bc54 8057ca62 nt!KeDelayExecutionThread+0x1c9 (FPO: [Non-Fpo])<br /> a9a5bc78 8057e7ff nt!IopCancelAlertedRequest+0x52 (FPO: [Non-Fpo])<br /> a9a5bc94 8057c341 nt!IopSynchronousServiceTail+0xe1 (FPO: [Non-Fpo])<br /> <span style="font-weight: bold;">a9a5bd38 805409ac nt!NtWriteFile+0x5d7 (FPO: [Non-Fpo])</span><br /> a9a5bd38 7c90eb94 nt!KiFastCallEntry+0xfc (FPO: [0,0] TrapFrame @ a9a5bd64)<br />WARNING: Frame IP not in any known module. Following frames may be wrong.<br /> 04dcfd9c 00000000 0x7c90eb94<br /></span><br />Here we can see the two hanging threads in NtWriteFile I already have seen in user space.<br /><br />I also was able to go that far without Bill's help - now comes the special magic:<br />The key to the success is lying in the IRP ( <a href="http://msdn2.microsoft.com/en-us/library/aa491631.aspx"><i>I/O request packet</i></a>) list!<br /><br />So let's have at the details of the IRPs:<br /><span style=";font-family:courier new;font-size:85%;" >1: kd> !irp <span style="color: rgb(255, 0, 0);">816ecde0</span><br />Irp is active with 5 stacks 5 is current (= 0x816ecee0)<br />No Mdl: System buffer=81e316a8: Thread 81a81020: Irp stack trace.<br /> cmd flg cl Device File Completion-Context<br />[...]<br />>[ 4, 0] 0 1 <span style="color: rgb(102, 102, 204);">81e36040 </span>81ca1798 00000000-00000000 pending<br /> *** ERROR: Module load completed but symbols could not be loaded for ser2pl.sys<br />\Driver\Ser2pl<br /> Args: 00000002 00000000 00000000 00000004<br />1: kd> !irp <span style="color: rgb(51, 204, 0);">816b3710</span><br />Irp is active with 5 stacks 5 is current (= 0x816b3810)<br />No Mdl: System buffer=821b62f0: Thread 8169da90: Irp stack trace.<br /> cmd flg cl Device File Completion-Context<br /></span><span style=";font-family:courier new;font-size:85%;" >[...]<br /></span><span style=";font-family:courier new;font-size:85%;" >>[ 4, 0] 0 1 <span style="color: rgb(204, 102, 204);">81e37040 </span>81e06a78 00000000-00000000 pending<br /> \Driver\Ser2pl<br /> Args: 00000001 00000000 00000000 00000004<br /></span><br />In both cases it is the Ser2pl driver (from prolific) that it not working it's queue with status pending.<br /><br />But now to Bill statement that two threads are accessing the same port:<br />Let's see were the data is directed to:<br /><br /><span style="font-size:85%;">1: kd> !object </span><span style="color: rgb(102, 102, 204);font-size:85%;" >81e36040 </span><span style="font-size:85%;"><br />Object: 81e36040 Type: (823b1958) Device<br /> ObjectHeader: 81e36028 (old version)<br /> HandleCount: 0 PointerCount: 3<br /> Directory Object: e10023d0 Name: </span><span style="color: rgb(255, 0, 0); font-weight: bold;font-size:85%;" >Serial3</span><span style="font-size:85%;"><br /><br />1: kd> !object </span><span style="color: rgb(204, 102, 204);font-size:85%;" >81e37040 </span><span style="font-size:85%;"><br />Object: 81e37040 Type: (823b1958) Device<br /> ObjectHeader: 81e37028 (old version)<br /> HandleCount: 0 PointerCount: 3<br /> Directory Object: e10023d0 Name: </span><span style="color: rgb(255, 0, 0); font-weight: bold;font-size:85%;" >Serial2</span><br /><br />Finally we can get the port mapping:<br /><span style=";font-family:courier new;font-size:85%;" >1: kd> !object \GLOBAL??<br />[...]<br /> 08 e1632920 SymbolicLink COM2<br />[...]<br /> e1789a58 SymbolicLink COM3<br />[...]<br /> e37b2e40 SymbolicLink COM4<br />[...]<br /> e383d438 SymbolicLink COM5<br /><br />1: kd> !object e1632920<br />Object: e1632920 Type: (823ed398) SymbolicLink<br /> ObjectHeader: e1632908 (old version)<br /> HandleCount: 0 PointerCount: 1<br /> Directory Object: e1000788 Name: COM2<br /> Target String is '\Device\Serial0'<br />1: kd> !object e1789a58<br />Object: e1789a58 Type: (823ed398) SymbolicLink<br /> ObjectHeader: e1789a40 (old version)<br /> HandleCount: 0 PointerCount: 1<br /> Directory Object: e1000788 Name: </span><span style="color: rgb(255, 0, 0);font-family:courier new;font-size:85%;" >COM3</span><span style=";font-family:courier new;font-size:85%;" ><br /> Target String is '\Device\</span><span style="color: rgb(255, 0, 0);font-family:courier new;font-size:85%;" >Serial2</span><span style=";font-family:courier new;font-size:85%;" >'<br />1: kd> !object e37b2e40<br />Object: e37b2e40 Type: (823ed398) SymbolicLink<br /> ObjectHeader: e37b2e28 (old version)<br /> HandleCount: 0 PointerCount: 1<br /> Directory Object: e1000788 Name: </span><span style="color: rgb(255, 0, 0);font-family:courier new;font-size:85%;" >COM4</span><span style=";font-family:courier new;font-size:85%;" ><br /> Target String is '\Device\</span><span style="color: rgb(255, 0, 0);font-family:courier new;font-size:85%;" >Serial3</span><span style=";font-family:courier new;font-size:85%;" >'<br />1: kd> !object e383d438<br />Object: e383d438 Type: (823ed398) SymbolicLink<br /> ObjectHeader: e383d420 (old version)<br /> HandleCount: 0 PointerCount: 1<br /> Directory Object: e1000788 Name: COM5<br /> Target String is '\Device\Serial4'</span><br /><br />So one thread was writing on to COM3 and the other to COM4. I was a bit less concerned that it was not obviously my fault but of course the problem is not gone. Now I hope to get fix from the cable supplier or select different hardware.<br />I must say I learned a lot with this case but it is not the type of issue I'm begging for!<br /><br />BTW: after all this investigations I found post on Marks blog about this class of defect: <a href="http://blogs.technet.com/markrussinovich/archive/2005/08/17/unkillable-processes.aspx">Unkillable Processes</a><br /><br />Finally a big thanks to Microsoft Support!Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com2tag:blogger.com,1999:blog-18989586.post-7129197711047491642008-02-12T09:52:00.000+01:002008-02-12T10:04:31.525+01:00Getting the GCRoot on a range of objectsRecently I was in the situation to look at the GCRoot of a bunch of objects in a range.<br />Eran <a href="http://dotnetdebug.net/2005/12/04/dumpobj-on-multiple-objects/">posted</a> on how to do this taks for getting detailed object information. I used this as a template for my needs:<br /><br /><span style=";font-family:courier new;font-size:85%;" >.foreach (obj { !DumpHeap -short <span style="color: rgb(51, 204, 0);">start_address end_</span></span><span style=";font-family:courier new;font-size:85%;" ><span style="color: rgb(51, 204, 0);">address</span> </span><start_address><end_address></end_address></start_address><span style=";font-family:courier new;font-size:85%;" > <start_address> <end_address> </end_address></start_address></span><span style=";font-family:courier new;font-size:85%;" >}) { !GCRoot ${obj} }</span><br /><br />Now replace <start_address style="color: rgb(51, 204, 0);"><end_address></end_address></start_address><span style=";font-family:courier new;font-size:85%;" ><span style="color: rgb(51, 204, 0);">start_address</span> </span>and<span style=";font-family:courier new;font-size:85%;" > <span style="color: rgb(51, 204, 0);">end_address</span> </span><start_address><end_address>with your adresses...</end_address></start_address>Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-90056400395009287932008-02-08T08:39:00.000+01:002008-02-08T08:58:31.573+01:00Rank four on Google for my Blog?Today I was curious and just <a href="http://www.google.com/search?q=windbg">googled for windbg.</a><br />Surprisingly I noticed, that my blog got rank 4:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVWetMWctD_any1D9ey_6h0VGpoWplYN9O9MCt0XSelfgLo-hxLRC9IDLTPaPg5TVzOvH4ydjWXZkV0agMAdhaq5JyR81pokoO8KOxm3NKsG7PuiDAQotyWMzLD7imLszRjZ9E/s1600-h/Windbg_Google_Rank.png"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVWetMWctD_any1D9ey_6h0VGpoWplYN9O9MCt0XSelfgLo-hxLRC9IDLTPaPg5TVzOvH4ydjWXZkV0agMAdhaq5JyR81pokoO8KOxm3NKsG7PuiDAQotyWMzLD7imLszRjZ9E/s320/Windbg_Google_Rank.png" alt="" id="BLOGGER_PHOTO_ID_5164514471141636354" border="0" /></a><br /><br />I do not understand this, because there are lots of blogs (<a href="http://blogs.msdn.com/tess/">Tess</a>, <a href="http://www.dumpanalysis.org/blog/">Dimtry</a>, <a href="http://www.wintellect.com/cs/blogs/jrobbins/default.aspx">John</a> just to name three of the best) outside that earn a much higher ranking than this one. The primary intent of this blog was to extend my brain which unfortunately is easily loosing things over time. So if you should hit my blog just googling for windbg make sure to visit the blogs on my roll...<br /><br />BTW: Tess is currently conducting an online <a href="http://blogs.msdn.com/tess/pages/net-debugging-demos-information-and-setup-instructions.aspx">training on windbg</a>. Check it out...Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com3tag:blogger.com,1999:blog-18989586.post-16591034620584433802008-01-31T10:26:00.000+01:002008-01-31T10:46:46.417+01:00Setting multiple breakpoints via wildcard patternSometimes I need a break point on a specific funtion in multiple classes. Examples are the use of templates, interfaces or inheritence.<br /><br />This can be easily achived via the bm (I translate as break match).<br /><br />Example:<br /><span style="font-size:85%;"><span style="font-family:courier new;"><br />bm /a MyModule!!CComCollectionMap*::*get_Exists*</span></span><br /><br />This will set a deferred breakpoint on every function that matches the given expression.<br />It is a good idea to check the matches upfront with the following expression:<br /><br /><span style="font-size:85%;"><span style="font-family:courier new;">x MyModule!!CComCollectionMap*::*get_Exists*</span></span><br /><br />In order to clear all currently set break points use:<br /><br /><span style="font-family: courier new;font-size:85%;" >bc *</span><br /><br />For more details refer to the very good Online Help under "bp, bu, bm (Set Breakpoint)"Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-16198075737220637652008-01-30T11:08:00.000+01:002008-01-30T13:52:08.445+01:00List source lines at current scope ipTo list the source at the currently selected frame (which can be set by either the .frame command or by clicking into the call stack window) use the lsa (List Source Lines) command:<br /><br /><span style="font-size:85%;"><span style="font-family:courier new;">0:000> lsa @$scopeip</span><br /><span style="font-family:courier new;"> 578: </span><br /><span style="font-family:courier new;"> 579: ' Setting device name</span><br /><span style="font-family:courier new;"> 580: If (gPrintContext.strPrinter = "") Then</span><br /><span style="font-family:courier new;"> 581: If (rpt.Printer.NDevices > 0) Then</span><br /><span style="font-family:courier new;">> 582: gPrintContext.strPrinter = rpt.Printer.DeviceName</span><br /><span style="font-family:courier new;"> 583: ' "Device Name is empty hence setting it to the default printer."</span><br /><span style="font-family:courier new;"> 584: Else</span><br /><span style="font-family:courier new;"> 585: ' the .PageSetup will show the system message</span><br /><span style="font-family:courier new;"> 586: ' "No printer installed."</span><br /><span style="font-family:courier new;"> 587: End If</span><br /><br /><br /></span>Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-16045282860061676572008-01-28T13:57:00.001+01:002008-02-08T08:58:16.938+01:00Google Custom Search for WinDbg<div>I often came across the point where I needed to search the web for a common phrases used in very special technical manner. One example is <a href="http://msdn2.microsoft.com/en-us/library/bb190764.aspx">sos</a> or son of strike. If you simply search for sos you will get everything but not what you are interested in.<br /><br />For getting help finding the needle in the hay when it comes to windbg I created a <a href="http://www.google.com/coop/cse/">Google Custom Search Engine</a> for windbg.<br /><br />You can find the CSE <a href="http://www.google.com/coop/cse?cx=018352668946159089029%3Ajx_rdg4340u&hl=en">here</a> or directly on my page here:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHPuest04L9GV64Eut1E1wp-iLDFgR716Ujq4DLUPi7n16YhxPNL2NczuzlYW32XA3V1yjej_Y5-Gf4krlnx2bBrAVbvArm5Y5fO22jlELtULtLzRLXGJV8C6539QzDs2_Bzy7/s1600-h/GoogleCustomSearchWindbg.png"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHPuest04L9GV64Eut1E1wp-iLDFgR716Ujq4DLUPi7n16YhxPNL2NczuzlYW32XA3V1yjej_Y5-Gf4krlnx2bBrAVbvArm5Y5fO22jlELtULtLzRLXGJV8C6539QzDs2_Bzy7/s320/GoogleCustomSearchWindbg.png" alt="" id="BLOGGER_PHOTO_ID_5160539126721870066" border="0" /></a><br /><br /><br />If you have additional sites for me to add, please drop a comment here....</div>Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com1tag:blogger.com,1999:blog-18989586.post-68891605229534324042008-01-24T09:59:00.000+01:002008-01-30T16:19:06.762+01:00CoCreateInstance: Which object in which dll<div>I came across a process that was consuming statically 25% CPU which I didn't understand why.<br />So I attached windbg and issued<br /><br /><span style="font-size:85%;">0:004> !runaway<br />User Mode Time<br />Thread Time<br /> 4:328 0 days 0:07:22.015<br /> 0:2a0 0 days 0:00:25.078<br /> 5:228 0 days 0:00:00.000<br /> 3:324 0 days 0:00:00.000<br /> 2:320 0 days 0:00:00.000<br />1:31c 0 days 0:00:00.000<br /></span><br />So switch to that thread<br /><br /><span style="font-size:85%;"><span style="font-family:courier new;">0:004> ~4 s</span></span><br /><br />And issue a stack dump:<br /><span style="font-size:85%;"><br /><span style="font-family:arial;">0:004> kb 100</span><br /><span style="font-family:arial;">ChildEBP RetAddr Args to Child</span><br /><span style="font-family:arial;">[...]</span><br /><span style="font-family:arial;">010be7a0 <span style="color: rgb(0, 0, 0);">77f6946c </span><span style="color: rgb(204, 51, 204);">010bec5c </span>00000000 00000401 ole32!</span><a style="font-family: arial;" href="http://msdn2.microsoft.com/en-us/library/ms686615%28VS.85%29.aspx">CoCreateInstance</a><span style="font-family:arial;">+0x37</span><br /><span style="font-family:arial;">[...]</span><br /><span style="font-family:arial;">010bfe7c <span style="color: rgb(0, 0, 0);">0052770f </span><span style="color: rgb(51, 204, 0);">010bfe98 </span>010bfee0 0052773f shell32!</span><a style="font-family: arial;" href="http://msdn2.microsoft.com/en-us/library/bb762154.aspx">ShellExecuteExA</a><span style="font-family:arial;">+0x203</span></span><br /><br />So I had two questions:<br /><br />1.) Which process should be launched by ShellExecute<br />2.) Which object should be created by CoCreateInstance<br /><br />ad 1.) First param passed to ShellExecute is of type <a href="http://msdn2.microsoft.com/en-us/library/bb759784%28VS.85%29.aspx">LPSHELLEXECUTEINFO</a><br />dt _SHELLEXECUTEINFO 010bfe98 didn't work for me although having symbol server setup correctly. So I needed the fifth pointer in the struct:<br /><br /><span style="font-size:85%;">0:004> dda <span style="color: rgb(51, 204, 0);">010bfe98 </span></span><br /><br />did the job:<br /><span style="font-size:85%;">010bfe98 0000003c<br />010bfe9c 00000440<br />010bfea0 00000000<br />010bfea4 00527768 "open"<br />010bfea8 00527758 "Viewer.exe" <== 010bfeac 00000000 </span><br />ad 2.) First param passed is of type REFCLSID or <a href="http://msdn2.microsoft.com/en-us/library/aa373931%28VS.85%29.aspx">GUID</a><br />Then I got it via:<br /><span style="font-size:85%;">0:004> dt ntdll!_GUID <span style="color: rgb(204, 51, 204);">010bec5c </span><br />{871c5380-42a0-1069-a2ea-08002b30309d}<br /> +0x000 Data1 : 0x871c5380<br /> +0x004 Data2 : 0x42a0<br /> +0x006 Data3 : 0x1069<br /> +0x008 Data4 : [8] "???"<br /></span><br />Now use either regedit or more elegantly:<br /><span style="font-size:85%;">0:005> !dreg hkcr\CLSID\{871C5380-42A0-1069-A2EA-08002B30309D}\InProcServer32\!*<br />Value: "" - REG_EXPAND_SZ: "%SystemRoot%\system32\shdocvw.dll"<br />expanded = "C:\WINDOWS\system32\shdocvw.dll"<br />------------------------------------------------------------------------<br />Value: "ThreadingModel" - REG_SZ: "Apartment"<br />------------------------------------------------------------------------<br /></span><br /><blockquote></blockquote>That's it!<br /></div>Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com2tag:blogger.com,1999:blog-18989586.post-5204748026428871892008-01-17T13:53:00.000+01:002008-01-30T16:20:20.611+01:00Setting a thread sensitive BreakpointI just came across the problem that I needed to break at a function just in case this function is executed in one specific thread.<br /><br />In Visual Studio 2008 (and also earlier) you can achieve this via Breakpoint filters:<br />1.) Right click on the Breakpoint and click on Filter...<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgc805L0jXYA0yI5QQQv3g7dmtyVzNMgcGLcSMEsM8pdiFJQo6jnmPY5V4ZUK1-CUJZZFhAUj4mDxFBWvR1oNTiFRrEK1Lb_-zKG7ZKcu1_noM-F6byZYc56pu2-mFPjeHVbud7/s1600-h/VisualStudioBPFilter1.png"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgc805L0jXYA0yI5QQQv3g7dmtyVzNMgcGLcSMEsM8pdiFJQo6jnmPY5V4ZUK1-CUJZZFhAUj4mDxFBWvR1oNTiFRrEK1Lb_-zKG7ZKcu1_noM-F6byZYc56pu2-mFPjeHVbud7/s320/VisualStudioBPFilter1.png" alt="" id="BLOGGER_PHOTO_ID_5156430337290362738" border="0" /></a><br /><br />2.) Now specify the thread id you want to break in...<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQlz6U6omUQbfb7xhkEK3xuQzn6gXCfbMI5nVjdq3zlVZp8YgJZ5klaUPu8bBcgMq5JcJVPUzz-uQCLkGgB-U7EFuyGQ4XFCSqDj8Nyqa5NVyORSqFyLZBwl2kEfjS_BpHufhj/s1600-h/VisualStudioBPFilter2.png"><img style="cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQlz6U6omUQbfb7xhkEK3xuQzn6gXCfbMI5nVjdq3zlVZp8YgJZ5klaUPu8bBcgMq5JcJVPUzz-uQCLkGgB-U7EFuyGQ4XFCSqDj8Nyqa5NVyORSqFyLZBwl2kEfjS_BpHufhj/s320/VisualStudioBPFilter2.png" alt="" id="BLOGGER_PHOTO_ID_5156430788261928834" border="0" /></a><br /><br />In windbg you can achieve it like this:<br /><br />Instead of typing:<br /><br /><span style="font-style: italic;">bp MyModule!MyClass::MyFunction+MyOffset</span><br /><br /><br />type:<br /><span style="font-style: italic;">~ 13 bp MyModule!MyClass::MyFunction+MyOffset</span><br /><br />to break in just if this function is executed in thread with id 13Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com0tag:blogger.com,1999:blog-18989586.post-507780563045399632008-01-17T09:56:00.000+01:002008-01-30T22:43:21.155+01:00Wishlist: Writing Debugger extension program in C#In xqiu's blog I found an interesting post about <a href="http://blogs.gotdotnet.com/xqiu/archive/2008/01/15/writing-debugger-extension-program-in-c.aspx">Writing Debugger extension program in C# </a>. Unfortunately the mentioned mdbeng.dll is not public yet. I contacted the Debugger Team with the wish to get it. Let's see what comes...Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com1tag:blogger.com,1999:blog-18989586.post-47531621576716910882007-10-30T15:18:00.000+01:002008-01-30T22:58:05.411+01:00Failed to load data access DLL, 0x80004005 - hmMe and some colleagues of mine recently all stumbled over the following error when analyzing .NET minidumps:<br />I opened a the minidump and typed !CLRStack. What I got was:<br /><br /><span style=";font-family:courier new;font-size:85%;" >Failed to load data access DLL, 0x80004005<br />Verify that 1) you have a recent build of the debugger (6.2.14 or newer)<br /> 2) the file mscordacwks.dll that matches your version of mscorwks.dll is<br /> in the version directory<br /> 3) or, if you are debugging a dump file, verify that the file<br /> mscordacwks_<arch>_<arch>_<version>.dll is on your symbol path.<br /> 4) you are debugging on the same architecture as the dump file.<br /> For example, an IA64 dump file must be debugged on an IA64<br /> machine.<br /><br />You can also run the debugger command .cordll to control the debugger's<br />load of mscordacwks.dll. .cordll -ve -u -l will do a verbose reload.<br />If that succeeds, the SOS command should work on retry.<br /><br />If you are debugging a minidump, you need to make sure that your executable<br />path is pointing to mscorwks.dll as well.</version></arch></arch></span><span style="font-size:85%;"><br /></span><br />OK, so I followed the step 1 through 4...<br />ad 1) I'm using version 6.7.5.0 for good <a href="http://voneinem-windbg.blogspot.com/2007/10/windbg-version-6840-is-out-im-still.html">reasons</a><br />ad 2) Don't know what that means...<br />ad 3) Why should a dll be in my symbol path?!?<br />ad 4) The architecture is x86_32 on both machines<br /><br />So I tried<span style="font-size:85%;"><span style="font-family:courier new;"> </span><span style="font-family:arial;"><span style="font-family:courier new;">.cordll -ve -u -l</span><br /><br /><span style="font-family:courier new;">0:000> .cordll -ve -u -l</span><br /><span style="font-family:courier new;">CLRDLL: Unknown processor type in image C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll</span><br /><span style="font-family:courier new;">CLR DLL status: No load attempts</span><br /><br /></span></span>The following sentence finally helped me:<br /><br /><span style="font-size:85%;"><span style="font-family:courier new;">If you are debugging a minidump, you need to make sure that your executable</span><br /><span style="font-family:courier new;">path is pointing to mscorwks.dll as well.</span><br /></span><br />So I executed:<br /><br /><span style="font-size:85%;"><span style="font-family:courier new;">0:000> lmv m mscorwks</span><br /><span style="font-family:courier new;">start end module name</span><br /><span style="font-family:courier new;">79e70000 7a3d6000 mscorwks T (pdb symbols) C:\windbg\symbols\mscorwks.pdb\6D3E0DE91A284256A48A60718DC9CDEB2\mscorwks.pdb</span><br /><span style="font-family:courier new;"> Loaded symbol image file: mscorwks.dll</span><br /><span style="font-family:courier new;"> Image path: C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll</span><br /><span style="font-family:courier new;"> Image name: mscorwks.dll</span><br /><span style="font-family:courier new;"> Timestamp: Fri Apr 13 09:15:54 2007 (461F2E2A)</span><br /><span style="font-family:courier new;"> CheckSum: 005635C7</span><br /><span style="font-family:courier new;"> ImageSize: 00566000</span><br /><span style="font-family:courier new;"> File version: 2.0.50727.832</span><br /><span style="font-family:courier new;"> Product version: 2.0.50727.832</span><br /><span style="font-family:courier new;"> File flags: 0 (Mask 3F)</span><br /><span style="font-family:courier new;"> File OS: 4 Unknown Win32</span><br /><span style="font-family:courier new;"> File type: 2.0 Dll</span><br /><span style="font-family:courier new;"> File date: 00000000.00000000</span><br /><span style="font-family:courier new;"> Translations: 0000.04b0 0000.04e0 0409.04b0 0409.04e0</span><br /></span><br />And...<br /><br /><span style="font-size:85%;"><span style="font-family:courier new;">0:000> .exepath+ C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\</span></span><br /><br />Finally...<br /><br /><span style="font-size:85%;"><span style="font-family:courier new;">0:000> .reload</span><br /><span style="font-family:courier new;">...................................................................................................................................................................................................................................................</span><br /><span style="font-family:courier new;">CLRDLL: Loaded DLL C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscordacwks.dll</span><br /></span><br />And my !CLRStack worked ;-)Volker von Einemhttp://www.blogger.com/profile/03633759507649031608noreply@blogger.com11