Reliable, Free P2V Solution

If you have ever had to take on a Physical-to-Virtual (P2V) migration of a server or other machine, you know that you generally have a fine balance of the following triangle of factors: cost, complexity, and reliability. It seems like the only way to get an easy and reliable solution is to throw gobs of money at either a VMWare infrastructure or something like Platespin. Don’t get me wrong: from what I’ve heard they both offer excellent solutions in that field, but it simply isn’t cheap. I had to find out the hard way, but there is another way.

The problem I was facing was that within my first month at a new job (still working there; awesome shop), I was tasked with a P2V conversion on a server, with no virtualization infrastructure in place in the organization. Since we were new to virtualization at the time and were just dabbling to start, keeping costs low (or non-existent) would also be a plus. I had to research and present a solution, figure out how it works, and get it done. We are very much a Microsoft shop, and the preference was for a Microsoft solution; so, Virtual Server it is. No problem there, as VS is free and so helped on the cost side. Now that we had the virtualization platform figured out, we just needed some way to get it from the physical box into virtual hard drives (VHD’s) with the whole thing still operational at the end. Here’s where the problems began.

Now, this was late 2006, and even in the last year virtualization has come a long way. Doing a little digging, however, I found out that Microsoft had actually released a free P2V too called the Virtual Server Migration Toolkit (VSMT). Naive as I was, I was excited that I had found an all-Microsoft solution that was still free and so fit the bill for the project. So…remember that triangle of cost/complexity/reliability that I was talking about? While VSMT is low on the cost, it cranks the complexity factor WAY up! I also discovered that the reliability portion was not really all there. If any of you have had the traumatic experience of working with VSMT, you’ll know what I’m talking about. For those fortunate souls unacquainted with it, here’s a brief glimpse into VSMT:

It requires Microsoft Automated Deployment Services (ADS), which is basically an MS solution for deploying Server 2003 to bare metal servers over the wire; fair enough. ADS in turn requires Windows Server 2003 Enterprise Edition and DHCP with a PXE environment. The gist is that the source machine (the machine you are trying to perform the P2V on) boots with a PXE image sent out by the ADS server. This puts the machine into a state where the ADS server controls it. At this point VSMT enters the picture. You then have to design what is best described as a workflow of various VBS and XML-based scripts that prepare the machine for cloning to an image, capture data from the machines hard disks to an image on the ADS server, again run some scripts against the image to take care of things like making the HAL and kernel generic, and then create a Virtual Server configuration file (.vmc) for the whole setup. Granted I was new to this game at that point, but I spent the better part 2 weeks on this project, with various failures ranging from the initial capture phase, blue screens on my ADS server, having to manually find/replace variables in scripts because they weren’t expanding properly, among other things, and all with virtually zero support available either from MS sites or from the community at large (as no one was apparently crazy enough to try to use the thing). I later attended a Technet conference on Microsoft’s virtualization solutions in Vancouver. When I mentioned to one of the presenters in a Q&A section that I had successfully completed a P2V using VSMT, he expressed actual surprise at the fact that someone got the whole thing to work! A few months later, the VM started logging disk errors and failed to back up successfully, and we ended up scrapping the VM and building a new one from scratch. Like I said, low cost but high complexity and low reliability.

So, when I heard that Microsoft was releasing a Beta of Systems Center Virtual Machine Manager (SCVMM, although it was just VMM at that point), I jumped at the chance to try this thing out. Point and click P2V? Sign me up! I later discovered that Beta1 did not yet include P2V functionality, so my hopes at a simple P2V solution faded again. These days, of course, you can get yourself a copy of the full version System Center Virtual Machine Manager for around $500 US, but unfortunately that still does not completely satisfy my “cheap-to-free” preference.

When the next P2V project rolled around, I was desperate to find something, anything that would avoid the VSMT debacle. I even started looking outside of the Microsoft camp to anything that could get our physical box running as a VM, and lo, I was rewarded! Enter VMWare Converter. VMWare had been kind enough to release a simple, free P2V utility that can run on pretty much any version of Windows, including client OS’s like XP, and was a simple wizard-driven point-and-click process without all the infrastructure requirements. Beautiful! Unfortunately this did not get us to a Virtual Server machine. But, wait! There’s hope! A simple, self-contained utility called VMDK2VHD was released by vmToolkit. This beauty of a tool allows you to convert your VMWare VMDK-format virtual hard drives into MS Virtual Server/PC VHD files. I was set!

The overall process was even pretty painless. Of course it can take a few hours for the whole process if you’re converting machines with a whole bunch of data, but trust me, it’s a breeze compared to some of the other options out there.

So, here’s what you need:

• Source physical machine
• I recommend having a conversion tools machine to install all the required software onto and to do all the grunt work of the actual conversions.
• VMWare Converter installed (probably on your conversion tools machine)
• Access to enough disk to store both the VMWare VMDK hard disk files and the Virtual Server/PC VHD hard disk files
• VMWare Server installed (probably on your conversion tools machine).
• PrepVM.vbs script (more on that later)
• Generic hal.dll or ntoskrnl.exe, either from your Windows setup CD or from %windir%\system32\servicepackfiles if you have applied a service pack at any time.
• VMDK2VHD extracted somewhere (probably on your conversion tools machine). Note that this utility is self-contained and does not need to be installed, just run.
• Virtual Server or Virtual PC installed somewhere to test the final product.

If you have all that, you are ready to go:

1. Ensure that the source physical machine is running, and that you have network connectivity to it from your conversion tools machine.
2. Launch the VMWare Converter application on your conversion tools machine.
3. Click Import Machine and follow the wizard to set the options for your conversion. A few notes: (1) VMWare Converter will install an agent on the source physical machine, and will ask you for administrative credentials to connect to the machine. (2) When selecting the partitions to convert, be sure to un-check utility partitions as are found on many servers. VMWare Converter will not understand these partitions, and the conversion will fail. (3) It is best to output your data to a single VMDK file and not to use the VMWare option of splitting the hard disk into multiple 2 GB files. Virtual Server does not use a similar technique, and the VMDK2VHD tool will not understand it as far as I recall. (4) Be sure to select the correct output format for your file. If you are running a different version of VMWare from the version that VMWare Converter is creating, you will not be able to start your VMWare virtual machine and complete the conversion process.
4. Start the conversion, and then go for lunch…this process can take a while, as it is reading all of your source data, converting it into a VMDK format, and writing it to the destination VMDK file.
5. When the conversion completes, I recommend making a backup copy of the VMDK files and working with those from now on. If things blow up, you can roll back to the VMDK files rather than having to re-do the whole capture process.
6. Configure a new virtual machine in VMWare Server to use the VMDK file created and start it up. I recommend leaving the network cards disconnected and logging on locally, as this will avoid problems such as IP address and naming conflicts that come with having both the physical machine and its virtual counterpart on the same LAN.
7. Log on to the VMWare virtual machine. NOTE: Cancel all of the “New Hardware Found” wizards that pop up. These are all for VMWare virtual devices, and we will be getting rid of these anyway. Installing them is just a waste of time and might mess with the whole process.
8. CAUTION BEFORE PROCEEDING: You will be replacing the hal.dll and ntoskrnl.exe files in %windir%\system32. I recommend making backup copies of the hal.dll and ntoskrnl.exe files at %windir%\system32. You don’t have to do this, but I prefer to err on the side of caution. ALSO: The replacement files MUST be of the same service pack level as your current machine. If the SP levels don’t match, your machine is very likely to give you hassles and might not even start. That being said…
9. Make the VM generic by copying the generic hal.dll and ntoskrnl.exe files from either your Windows setup disk or from the %windir%\system32\servicepackfiles directory. If you are using the service pack files, you can simply copy the files into the VM’s %windir%\system32 folder. If you are using the Windows setup disk, you will need to run the following commands on the server (assuming that your CD drive on the machine is D:\): expand d:\i386\hal.dl_ %windir%\system32\\hal.dll and expand d:\i386\\ntoskrnl.ex_ %windir%\system32\ntoskrnl.exe.
10. Uninstall VMWare Tools if it has been installed.
11. Run the Prepvm.vbs script included at the end of this page. The script disables the VMWare devices in the machine so that they are not present when the machine starts up in Virtual Server/PC. It will also shut down the VM for the final conversion process when it’s done running.
12. At this point, you could make more backup copies of the VMDK files, but, to be honest, the “make the machine generic” steps are fairly quick, and will take much less time to re-do than to copy gigs and gigs of VMDK files as backups.
13. Launch the VMDK2VHD utility on your conversion tools machine. The interface is very simple, and basically just asks you for the source VMDK file and the destination VHD file that you want to output it to. It also does ask you if you want a fixed or dynamically-expanding VHD file. This is totally up to you. Basically, think of it this way: If your VHD represents an 80 GB hard disk, but you only have 10 GB of data on it, a fixed-size disk will be 80 GB in size, whereas a dynamically-expanding VHD will be 10 GB in size and will grow as the size of the data on your VHD grows.
14. Let the utility run, and go and grab a coffee/dinner/breakfast/whatever, because this will again take a fair bit of time, depending on the amount of data you’re working with.
15. When the conversion is done, configure a new VM in Virtual Server/PC that uses the VHD file produced by VMDK2VHD as your virtual hard disk.
16. Start up the Virtual Server/PC VM, again probably with NIC’s disconnected to avoid IP/Naming conflicts.
17. Logon to the machine and install all of the hardware that Windows finds.
18. If everything seems in order, celebrate! Shut down your source physical machine, connect the VM to the network, and away you go!

Again, this post would not be possible without other smarter people having gone before me. Some resources:

How to convert VMware virtual disk images to Virtual Server

Prepvm.vbs script