Accessing VMFS partitions on an ESX server that doesn’t boot normally

This week I was dealing with an ESX vSphere v4.0.0 that wasn’t booting properly and had problem with mounting VSD.

it was showing the following message:

“VSD Mount Failed


You have entered the recovery shell. You can try booting in troubleshooting mode to attempt to fix the issue. If that fails, you should contact VMWare techsupport.

/bin/sh: can´t access tty: job control turned off.”

All I wanted to do was to transfer all the Virtual Machines (all VMDK files) out of that server but, because it was in recovery shell, I wasn’t able to mount an external disk or connect via SSH.

After a lot of googling and testing, I found a way of doing it.  This method is useful for disaster recovery as well as forensic analysis in case you have to image an ESX server.  Forensic software like Encase and FTK don’t support VMFS file system and when you add an image of an ESX server as evidence, they just show the VMFS partitions as Unknown.

These are the steps for mounting VMFS partitions under Linux / Unix:

1.  vmfs-tools is a tool which is “originally loosely based on the vmfs code from fluidOps” and allows read only access to VMFS file systems from non ESX/ESXi hosts.

2.  boot the ESX server with a live linux CD (I used Ubuntu 10 ) and when it asks if you want to install or just try it live, choose try.

3.  after the OS is booted and you can see the desktop, connect the OS to Internet and install vmfs-tools by running: sudo apt-get install vmfs-tools ( if it says it can’t find the package, go to software source setting and add universe by checking the box).  Alternatively, if you don’t have internet connection, you can download the vmfs-tools package from here and put it on a usb and install it manually using dpkg -i command (example dpkg -i vmfs-tools.deb).

4.  Now type  the following command: sudo fdisk –l this shows that the vmfs file system is located at /dev/sdb3 ( if it’s a server with SCSI raid unit, it will be under /dev/cciss/c0d0p#)

5.  The next command is to mount VMware VMFS partition:
create a folder as a mount point using this command mkdir /media/VMFS and use this command to mount the VMFS partition vmfs-fuse /dev/sdb3 /media/VMFS
6.  now go to that folder and check the content using cd /media/VMFS and then ls -hal

7.  you should be able to see all the VM folders.

 
8.  to export those VMs to an external disk, connect a USB disk and mount it. then use cp -r command to copy each VM folder to USB disk. (example cp -r /media/VMFS/VMname/ /Media/sdb1(usb disk partition mount point)/
 

 
I hope you have found this post useful.
 

 
Hadi Rahnama
Senior Digital Forensic Analyst
Vincents Forensic Technology