Thursday, February 12, 2009

Keeping the Pearl

I recently get a memory dump of a hanging process. So I opened it with windbg

The main thread was looking like this:

0:00> ~0 kb
ChildEBP RetAddr Args to Child
0012c72c 7c90e9c0 7c91901b 0000056c 00000000 ntdll!KiFastSystemCallRet
0012c730 7c91901b 0000056c 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0012c7b8 7c90104b 0097e4c0 7c910945 7c97e4c0 ntdll!RtlpWaitForCriticalSection+0x132
0012c7c0 7c910945 7c97e4c0 ffffffff 00000000 ntdll!RtlEnterCriticalSection+0x46
0012c800 7c80e2cf ffffffff 7c80e038 00000000 ntdll!RtlAcquirePebLock+0x28
0012c868 7c80e00d 7c80e038 00020690 00000000 kernel32!BasepComputeProcessPath+0x5f
0012c8a8 7c801b83 00000000 00000000 0012d7a0 kernel32!BaseComputeProcessDllPath+0xe3
0012c908 7c801d6e 7ffdfc00 00000000 00000000 kernel32!LoadLibraryExW+0x12f
0012c91c 7c801da4 06057954 00000000 00000000 kernel32!LoadLibraryExA+0x1f
0012c938 060b4823 06057954 7c911993 03f41ea8 kernel32!LoadLibraryA+0x94

Sieextpub's very usefull !critlist extension showed me were whose holding the critical section:

0:000> !critlist
CritSec at 7c97e4c0. Owned by thread 6.
Waiting Threads: 0 21
CritSec at 7c97c0d8. Owned by thread 21.
Waiting Threads: 22

SOS!Threads told me that it's a managed thread from the thread pool:

0:006> !Threads
ThreadCount: 5
UnstartedThread: 0
BackgroundThread: 5
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
PreEmptive GC Alloc Lock
ID OSID ThreadOBJ State GC Context Domain Count APT Exception
0 1 ea4 001886e0 4220 Enabled 00000000:00000000 0019c7b8 1 STA System.Runtime.InteropServices.SEHException (0177d128)
2 2 2c0 001a6210 b220 Enabled 00000000:00000000 0019c7b8 0 MTA (Finalizer)
3 3 d7c 001efbd0 80a220 Enabled 00000000:00000000 0019c7b8 0 MTA (Threadpool Completion Port)
4 4 894 00223760 880b220 Enabled 00000000:00000000 0019c7b8 0 MTA (Threadpool Completion Port)
6 5 650 00224310 180b220 Enabled 017d2770:017d4608 0019c7b8 0 MTA (Threadpool Worker)

Now I wanted to get the Stack Dump for thread number 6:

0:006> !CLRStack
OS Thread Id: 0x650 (6)
0522f3bc 7c913da4 [NDirectMethodFrameStandalone: 0522f3bc]
0522f3d4 79364aef
0522f8f8 793644ae
0522f900 793821b0
0522f90c 793820f6
0522f928 79381dbe
0522f950 79381d16
0522f968 793a2014
0522f980 04189669
0522f9a0 041892b2
0522fa08 0418916f
0522fa38 79407caa
0522fa3c 79373ecd
0522fa54 79407e18
0522fa68 79407d90
0522fbf8 79e7c74b [GCFrame: 0522fbf8]

Oops? OK - sometimes !DumpStack does a better job:

0:006> !DumpStack
OS Thread Id: 0x650 (6)
Current frame: ntdll!RtlGetFullPathName_Ustr+0x653
ChildEBP RetAddr Caller,Callee
0522f2d0 79e75002 mscorwks!FrameWithCookie<HelperMethodFrame>::FrameWithCookie<HelperMethodFrame>+0x1e, calling mscorwks!HelperMethodFrame::HelperMethodFrame
0522f318 7c91056d ntdll!RtlFreeHeap+0x647, calling ntdll!_SEH_epilog
0522f33c 7c9141e4 ntdll!RtlGetFullPathName_U+0x33, calling ntdll!RtlGetFullPathName_Ustr
0522f364 7c80b86c kernel32!GetFullPathNameW+0x1a, calling ntdll!RtlGetFullPathName_U
0522f37c 015da1c3 015da1c3
0522f3a4 79364aef 79364aef, calling 796c00cc
0522f3c4 79364aef 79364aef, calling 796c00cc
0522f8f0 793644ae 793644ae, calling 793644b8
0522f8f8 793821b0 793821b0, calling 79364480
0522f904 793820f6 793820f6, calling 79382120
0522f920 79381dbe 79381dbe, calling 79381fac
0522f934 79381d16 79381d16, calling 79381d54
0522f95c 793a2014 793a2014, calling 79381cd4
0522f978 04189669 04189669
0522f994 041892b2 041892b2, calling 04189600
0522fa00 0418916f 0418916f, calling 04189208
0522fa30 79407caa 79407caa
0522fa34 79373ecd 79373ecd
0522fa48 79407e18 79407e18, calling 79373e4c
0522fa60 79407d90 79407d90, calling 79407dc8
0522fa74 79e7c74b mscorwks!CallDescrWorker+0x33
0522fa84 79e7c6cc mscorwks!CallDescrWorkerWithHandler+0xa3, calling mscorwks!CallDescrWorker
0522fb04 79f00eca mscorwks!DispatchCallBody+0x1e, calling mscorwks!CallDescrWorkerWithHandler
0522fb24 79f00e75 mscorwks!DispatchCallDebuggerWrapper+0x3d, calling mscorwks!DispatchCallBody
0522fb88 79f00f03 mscorwks!DispatchCallNoEH+0x51, calling mscorwks!DispatchCallDebuggerWrapper
0522fbbc 79f6e39d mscorwks!QueueUserWorkItemManagedCallback+0x6c, calling mscorwks!DispatchCallNoEH
0522fc1c 79ef3207 mscorwks!Thread::DoADCallBack+0x32a
0522fc30 79ef31a3 mscorwks!Thread::ShouldChangeAbortToUnload+0xe3, calling mscorwks!Thread::DoADCallBack+0x2db
0522fc58 79e744d0 mscorwks!Thread::EnterRuntimeNoThrow+0x94, calling ntdll!RtlSetLastWin32Error
0522fc5c 79e744d7 mscorwks!Thread::EnterRuntimeNoThrow+0x9b, calling mscorwks!_EH_epilog3
0522fcc4 79ef30c3 mscorwks!Thread::ShouldChangeAbortToUnload+0x30a, calling mscorwks!Thread::ShouldChangeAbortToUnload+0x32
0522fd00 79ef4826 mscorwks!Thread::ShouldChangeAbortToUnload+0x33e, calling mscorwks!Thread::ShouldChangeAbortToUnload+0x29d
0522fd28 79ef48cf mscorwks!ManagedThreadBase::ThreadPool+0x13, calling mscorwks!Thread::ShouldChangeAbortToUnload+0x319
0522fd40 79f6e2d8 mscorwks!ManagedPerAppDomainTPCount::DispatchWorkItem+0xdb, calling mscorwks!ManagedThreadBase::ThreadPool
0522fd94 79f022c7 mscorwks!PerAppDomainTPCountList::GetAppDomainIndexForThreadpoolDispatch+0x35
0522fda8 79f0202a mscorwks!ThreadpoolMgr::ExecuteWorkRequest+0xaf
0522fdbc 79f021a0 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x223, calling mscorwks!ThreadpoolMgr::ExecuteWorkRequest
0522fe14 79f95a2e mscorwks!Thread::intermediateThreadProc+0x49
0522ffa0 79f95a1c mscorwks!Thread::intermediateThreadProc+0x37, calling mscorwks!_alloca_probe_16
0522ffb4 7c80b683 kernel32!BaseThreadStart+0x37

Not really better, eh?

So I launched my good old (which lives side by side with the latest windbg on my machine - thanks to xcopy) and surprise:

[click image to enlarge]


The call stack window did a wonderful job and I even got pointed to the corresponding source code.

As it is the same sos, a !DumpStack did not do the job in either.

So please Microsoft, give us back the this very very helpful feature of!
Or at least make SOS working like the stack window of


Anonymous said...

Is there a way, maybe, you "drop" this version somewhere ? :)

Volker von Einem said...

You might want to look here...

Volker von Einem said...

By the way - if you should find it somewhere: The md5 of MY copy is 4be556e2662fe182ee238d7f4d68ff84 *dbg_x86_6.7.05.0.exe