Thankfully XenServer is on the way out in my environment. One of the perpetual troubles is the issue with hung VM’s. The server blades are running XenServer 5.6 Service Pack 2.

Seemingly at random a VM will hang, console is unresponsive and SSH connections are nonexistent. Selecting “Force Reboot” or “Shutdown” operation in XenCenter does nothing at all to the VM. Forcing VM reboot from Xen host command line using xe vm-reboot vm=vm-prd-0001 force=true does not work either. The only way to fix this is to connect to XenServer and destroying the VM.

At the Xen host console get the UUID of the hung VM:

[root@blade0001 log]# xe vm-list | grep -A 2 -B 2 -i vm-prd-0001

uuid ( RO)           : 66edb0e7-2a87-5ceb-3595-f3c45526acf7
     name-label ( RW): vm-prd-0001
    power-state ( RO): running

[root@blade0001 log]#

Then find out domid of the VM:

[root@blade0001 log]# list_domains
 id |                                 uuid |  state
  0 | cc3d16f0-eb06-40ca-a395-fcc2928177fe |     R
 58 | f6a4db7f-b562-331d-754a-dca7e9f32a32 |    B H
 62 | 79e8b961-44bb-39a4-5769-98ba0fecad0e |    B  
 66 | c59bc020-f41d-7fec-3362-355fcda9427e |    B  
 68 | 0ab05a44-54fc-7466-046d-62d81cc8464d |    B  
 69 | 2429f1e5-8711-b10c-2758-021f9f1fe8f1 |    B  
 70 | 0580343a-cabc-1011-d95c-367b0d1d102c |    B  
106 | 4a855d78-410c-c8ae-022c-176a3a15cf3a |    B  
108 | e7106f78-089d-55ab-2495-12660a9efb07 |    B H
109 | 66edb0e7-2a87-5ceb-3595-f3c45526acf7 |      H
110 | 702fac93-a06c-e7e0-ebe1-9bdc73887ea2 |     R
111 | 07760d0b-aa30-ec4b-d375-5e7b6e5b6d00 |    B  
112 | 09715eec-2215-3309-d80e-ee521ae5c167 |    B H
120 | d71299ae-829c-e2b5-7780-f0e9473a5b64 |    B  
121 | bc4cfd91-8ad5-150a-5924-b321fd20490c |    B
[root@blade0001 log]#

In my case the domid is 109. Now that you have domid you will have to destroy it:

[root@blade0001 log]# /opt/xensource/debug/xenops destroy_domain -domid 109
[root@blade0001 log]# xe vm-reboot vm=vm-prd-0001 force=true
[root@blade0001 log]#

I think there might be Citrix knowledge base article for this, but I am not sure where it is at.