VMware Virtual Machine(VM) Remote Debugging using WinDBG

Instructions to setup WinDBG to debug VM in Kernel mode(on Windows XP):

Step 1: Edit the settings of you virtual machine to use a named pipe for a COM port

Edit
Settings
COM1
Named pipe:
Add: \\.\pipe\vpcdebug

Step 2: Edit boot.ini file on the virtual machine.

Right-click on My Computer
Properties
Settings
Edit start-up file manually:

Add: /DEBUG /DEBUGPORT = COM1 /BAUDRATE=115200
I like to add a second line here instead. Copy the last line, and append it with the above line:
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional" /noexecute=optin /fastdetect
/DEBUG /DEBUGPORT=COM1 /BAUDRATE=115200

This will give you two different boot options, one for debugging and the other for normal use.

Step 3: Set up WinDbg for remote debugging over COM1.
This can be done through the File -> Kernel Debug menu, but I prefer to make a shortcut so I can have multiple different types of debugging environments.

Create a shortcut that points to WinDbg.exe and add the the necessary command-line arguments. Your shortcut target should look something like this:
"C:\Program Files\Debugging Tools for Windows\windbg.exe" -k com:pipe,port=\\.\pipe\vpcdebug,resets=10

This is all you need to do to set up kernel debugging. Be sure to start the VM before WinDbg so that it has a chance to create the named pipe. Also, make sure you download the correct symbols for your environment. For example:

SRV*c:\websymbols* http://msdl.microsoft.com/download/symbols

Instructions to setup WinDBG to debug VM in Kernel mode(on Vista and above):

debug kernel

/debug /debugport=com1 /baudrate=115200

for Vista

bcdedit

bcdedit /copy {current} /d "DebugBoot"

(note new guid)

bcdedit /displayorder {current} {guid}

bcdedit /debug {guid} ON

On the "debugging server" (the PC with the problem)

Start WinDBG

Type .server npipe:pipe=pipename

On the "debugging client" (the one you're sitting at)

Start WinDBG

Go to File>Connect to Remote Session

For Connection String enter npipe:server=Server, pipe=PipeName [,password=Password] where Server is the hostname/IP and PipeName is the name that the server chose. The password section is optional.

Miscellaneous:

Use “CONTROL+BREAK” command if “Debuggie is not connected” is shown …

If the VM looks like HANG/LOCKED after starting Kernel debugging then type “g” in the WinDbg commandline so that debugging starts and we get the VM unlocked J

ref:

Remote Kernel Debugging - http://www.wd-3.com/archive/RemoteDbg.htm

WinDbg remote debugging and Vmware - http://silverstr.ufies.org/lotr0/windbg-vmware.html

http://www.ndis.com/ndis-debugging/virtual/vmwaresetup.htm