Boot a Windows Partition From Linux Using KVM

12 Dec 2021 10 min read

This guide explains how to boot a physical Windows partition inside linux using qemu/kvm and virt-manager. It extends the three tutorials (in particular jianmin’s tutorial) linked in the references section below and tries to provide more detaild step-by-step instructions.

This guide proceeds in tiny steps so that we can ensure that the changes are working after each step. In the first part of the guide all the commands are run using qemu on the command line and only at the end everything is tied together and a VM is configured in Virt Manager. Each Step has its own references section with links to other guides usful to better understand the instructions in that Step. Some Steps that are not required are crossed out. They have been kept as a reference to things not to do. The commands have been tested on Manjaro linux.

References

https://jianmin.dev/2020/jul/19/boot-your-windows-partition-from-linux-using-kvm/

https://k3a.me/boot-windows-partition-virtually-kvm-uefi/

https://lejenome.tik.tn/post/boot-physical-windows-inside-qemu-guest-machine

Random concepts

Virtio drivers

Paravirtualized devices, i.e. almost bare metal performance

Any virtio requires driver both on the host and guest

Graphics

GPU passthrough

Ideal case, the GPU is completely handled by the guest VM.

“Simple” to do with a dual GPU, one is passed the other is used by the host

Possible with a single GPU with unbinding, but cumbersome

https://czak.pl/2020/04/09/three-levels-of-qemu-graphics.html

Looking glass

https://looking-glass.io/

Solves some problems of the passthrough. Alpha code.

Virtio (virgl)

https://ryan.himmelwright.net/post/virtio-3d-vms/

Useful for Linux guest, because it is only useful for OpenGL (no DirectX)

Install KVM and QEMU

  1. Follow this guide to check that:

    • virtualization is supported and enabled in the bios
    • kvm kernel modules are loaded

  2. Install the required packages

    1. Install the basic packages

      1sudo pacman -S qemu virt-manager dnsmasq iptables-nft

    2. Install the UEFI firmware

      1sudo pacman -S edk2-ovmf

  3. Add our user to the kvm and libvirt groups

    1 sudo gpasswd -a $(whoami) kvm
2 sudo gpasswd -a $(whoami) libvirt

  4. Run and enable boot up start libvirtd daemon

    1sudo systemctl enable --now libvirtd

References

https://gist.github.com/diffficult/cb8c385e646466b2a3ff129ddb886185

Boot a physical Windows partition with UEFI

General notes

  • Adding/removing devices, cdrom, etc. may require changing the boot order (follow the troubleshoot in Step 4)
  • Sometimes Windows forgets to load the VirtIO disk driver and give you an INACCESSIBLE_BOOT_DEVICE repeat the procedure in Step 6 to make it happy

Step 1 - Create a virtual disk that includes the Windows partition

  1. Create two partitions for the efi bootloader

    1dd if=/dev/zero of=efi1 bs=1M count=100
2dd if=/dev/zero of=efi2 bs=1M count=1

  2. Identify the name of the Windows partition, e.g., /dev/nvme0n1p3

  3. Create a script start_md0 to build the disk

     1#!/usr/bin/env bash
 2WIN=/dev/nvme0n1p3
 3EFIDIR=$(cd $(dirname "${BASH_SOURCE[0]}") && pwd)
 4
 5set -e
 6
 7if [[ -e /dev/md0 ]]; then
 8  echo "/dev/md0 already exists"
 9  exit 1
10fi
11
12if mountpoint -q -- "${WIN}"; then
13  echo "Unmounting ${WIN}..."
14  umount ${WIN}
15fi
16modprobe loop
17modprobe linear
18LOOP1=$(losetup -f)
19losetup ${LOOP1} "${EFIDIR}/efi1"
20LOOP2=$(losetup -f)
21losetup ${LOOP2} "${EFIDIR}/efi2"
22mdadm --build --verbose /dev/md0 --chunk=512 --level=linear --raid-devices=3 ${LOOP1} ${WIN} ${LOOP2}
23chown $USER:disk /dev/md0
24echo "$LOOP1 $LOOP2" > ~/.win10-loop-devices

  4. Create a script stop_md0

    1#!/usr/bin/env bash
2mdadm --stop /dev/md0
3xargs losetup -d < /root/.win10-loop-devices

  5. Test the scripts

    1sudo ./start_md0
2ls /dev/md0
3sudo ./stop_md0

Step 2 - Create a partition table on the virtual disk

 1sudo parted /dev/md0
 2(parted) unit s
 3(parted) mktable gpt
 4(parted) mkpart primary fat32 2048 204799    # depends on size of efi1 file
 5(parted) mkpart primary ntfs 204800 -2049    # depends on size of efi1 and efi2 files
 6(parted) set 1 boot on
 7(parted) set 1 esp on
 8(parted) set 2 msftdata on
 9(parted) name 1 EFI
10(parted) name 2 Windows
11(parted) quit

Step 3 - Write a BCD entry on the UEFI boot menu

  1. Download a Windows DVD 10 ISO (free)

  2. Boot the DVD

    1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw \
4    -cpu host -enable-kvm -m 2G \
5    -cdrom /media/dataHD/system/kvm-windows/win10iso/Win10_21H1_English_x64.iso

  3. Click Shift + F10 to open a shell

  4. Assign a letter to the EFI partition

    1diskpart
2DISKPART> list disk
3DISKPART> select disk 0    # Select the disk
4DISKPART> list volume      # Find EFI volume (partition) number
5DISKPART> select volume 2  # Select EFI volume
6DISKPART> assign letter=B  # Assign B: to EFI volume
7DISKPART> exit

  5. Write a Boot Configuration Data (BCD) entry

    1bcdboot C:\Windows /s B: /f ALL

Step 4 - Boot the system the first time

1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw \
4    -cpu host -enable-kvm -m 2G

Troubleshoot

failed to load boot0001 uefi qemu dvd-rom qm00003 arch bvdboot

  1. Boot the VM
  2. Click Esc when you see the Tiano Core logo
  3. From Boot maintenance > Boot options change the boot order and put the disk on top

References

Step 5 - Install spice guest tools (with virtio drivers)

  1. Install spice guest tools inside the Windows VM
  2. Reboot

Step 5 - Install VirtIO guest additions

[Virtio drivers are included in the spice guest tools]

  1. Install virtio-win from AUR
  2. Boot the VM with the VirtioIO DVD
1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw \
4    -cpu host -enable-kvm -m 2G \
5    -cdrom /var/lib/libvirt/images/virtio-win.iso
  1. Open the dvd drive and install virtio-win-guest-tools

Step 5b - Load the virtio tablet driver

[No need to add the tablet device if we use spice]

Add a device to make the mouse work properly with the new graphics driver (it also prevents the mouse to be grabbed)

1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G \
5    -vga virtio -display sdl,gl=on \
6    -device virtio-tablet,wheel-axis=true

Step 6 - Boot Windows with disk virtio driver

Perform all the steps in this precise order.

Careful, you may need to touch the boot order after some of the steps because the virtio disk is seen as a different disk.

  1. Boot Windows with the virtio disk driver, Windows should look for the driver in the cdrom and figure out it should load it at boot time. If it fails proceed with steps 2-8

    1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G \
5    -cdrom /var/lib/libvirt/images/virtio-win.iso

  2. Create a dummy hard drive

    1qemu-img create -f qcow2 dummy.qcow2 1G

  3. Boot the system with the new hard drive using virtio drivers

    1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw \
4    -cpu host -enable-kvm -m 2G \
5    -cdrom /var/lib/libvirt/images/virtio-win.iso \
6    -drive file=dummy.qcow2,if=virtio

  4. In Windows run diskmgmt.msc

    1. check that the 1G disk is detected
    2. Right click on the disk > Properties and check that the driver is virtio

  5. Set Windows to boot in safe mode so that it loads all the driver at boot time

    1bcdedit /set {current} safeboot minimal

  6. Restart the VM with the md0 disk set to use the virtio drivers (keep the dummy disk and the cdrom)

    1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G \
5    -cdrom /var/lib/libvirt/images/virtio-win.iso \
6    -drive file=dummy.qcow2,if=virtio

  7. Set Windows to boot in normal mode

    1bcdedit /deletevalue {current} safeboot

  8. Start Windows without the dummy disk but with the cdrom

    1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G \
5    -cdrom /var/lib/libvirt/images/virtio-win.iso

  9. Start Windows without the cdrom

    1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G

References

https://wiki.archlinux.org/title/QEMU#Change_existing_Windows_VM_to_use_virtio

https://superuser.com/a/1253728

Step 7 - Load the qxl graphics driver

  1. Load the qxl vga driver
1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G \
5    -vga qxl -display sdl

Step 7 - Load the virtio graphics driver

[The virtio graphics driver aka virgl is only good for OpenGL not for DirectX, not so useful in Windows. It makes the mouse lag with spice]

  1. Load the virtio vga driver and add a device to make the mouse work properly with the new graphics driver (it also prevents the mouse to be grabbed)

    1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G \
5    -device virtio-tablet,wheel-axis=true \
6    -vga virtio -display sdl
7

  2. In Windows open Device Manager and check that the Display Adapter is a Virtio GPU

  3. Enable hardware acceleration

    1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G \
5    -device virtio-tablet,wheel-axis=true \
6    -vga virtio -display sdl,gl=on

  4. Check that the hardware acceleration is working by running dxdiag and looking at the second page

References

https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceleration

Step 8 - Load the networking virtio driver

1qemu-system-x86_64 \
2    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
3    -drive file=/dev/md0,media=disk,format=raw,if=virtio \
4    -cpu host -enable-kvm -m 2G \
5    -vga virtio -display sdl,gl=on \
6    -device virtio-tablet,wheel-axis=true \
7    -nic user,model=virtio-net-pci

Step 9 - Optimize the performance

  1. Tweaks:

    1. add cpu tweaks for Windows hypervisor
    2. define the cpu topology, smp 1 socket, 2 cores, 2 threads each (host system has a quad core cpu)
    3. extend the memory to 6 GB (host system has 16 GB of RAM)
    4. add virtio-balloon to reclaim memory from the guest
     1qemu-system-x86_64 \
 2    -enable-kvm \
 3    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
 4    -drive file=/dev/md0,media=disk,format=raw,if=virtio,aio=native,cache=none \
 5    -cpu host,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time \
 6    -smp 4,sockets=1,cores=2,threads=2 \
 7    -m 6G \
 8    -vga qxl -display sdl,gl=on \
 9    -nic user,model=virtio-net-pci \
10    -device virtio-balloon

  2. Enable huge pages (follow this guide to compute the number of pages and allow them to be used by the virtual machine)

     1qemu-system-x86_64 \
 2    -enable-kvm \
 3    -bios /usr/share/ovmf/x64/OVMF_CODE.fd \
 4    -drive file=/dev/md0,media=disk,format=raw,if=virtio,aio=native,cache=none \
 5    -cpu host,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time \
 6    -smp 4,sockets=1,cores=2,threads=2 \
 7    -m 6G \
 8    -mem-path /dev/hugepages \
 9    -vga virtio -display sdl,gl=on \
10    -device virtio-tablet,wheel-axis=true \
11    -nic user,model=virtio-net-pci \
12    -device virtio-balloon

    1. Set the number of huge pages to use before the vm starts

      1su -c "echo 3080 > /proc/sys/vm/nr_hugepages"

    2. Start the virtual machines and check that the HugePages_Free is less than HugePages_Total to ensure that hugepages are in use by the VM

      1grep HugePages /proc/meminfo

    3. Release the huge pages when the vm terminates

      1su -c "echo 0 > /proc/sys/vm/nr_hugepages"

  3. Enable tap networking

    • TODO (really needed? my network performance are already very good)

References

https://wiki.archlinux.org/title/QEMU#Improve_virtual_machine_performance

http://kvmonz.blogspot.com/p/knowledge-disk-performance-hints-tips.html

https://wiki.archlinux.org/title/KVM#Enabling_huge_pages

Step 10 - Configure a virt-manager VM

  1. [Wizard step 1] Select Import existing disk image
  2. [Wizard step 2]
    1. Existing storage path: /dev/md0
    2. Select the operating systems: win10
  3. [Wizard step 3] Choose the amount of memory and cpus to allocate
  4. [Wizard step 4] Check Customize configuration before install
  5. Overview
    1. Firmware: OVMF_CODE.fd
  6. CPUs
    1. Topology 1,2,2 (host system has a quad core cpu)
  7. Disk
    1. Remove the Sata Disk
    2. Add a Storage with Device Type equal to virtio
  8. NIC
    • Device model: virtio
  9. Tablet, remove this
  10. Video QXL
    • Do not use virtio
  11. Ensure all qemu arguments for tweaking performance are present in libvirt

    1. [auto] Disk parameters aio=native,cache=none

      1<driver name="qemu" type="raw" cache="none" io="native"/>

    2. [auto] Hypervisor enlightment hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time maps to:

       1<features>
 2 <hyperv>
 3   <relaxed state='on'/>
 4   <vapic state='on'/>
 5   <spinlocks state='on' retries='8191'/>
 6 </hyperv>
 7<features/>
 8
 9<clock ...>
10 <timer name='hypervclock' present='yes'/>
11</clock>

    3. [auto] smp, we defined the topology in a previous step

    4. Huge pages. Execute sudo virsh edit win10 and add the following block before the closing </domain> tag

      1<memoryBacking>
2  <hugepages/>
3</memoryBacking>

      Start the virtual machines and check that the free huge pages are less than the total hugepages

      1grep HugePages /proc/meminfo

References

https://wiki.libvirt.org/page/FAQ#What_is_the_difference_between_qemu:.2F.2F.2Fsystem_and_qemu:.2F.2F.2Fsession.3F_Which_one_should_I_use.3F

https://www.linuxquestions.org/questions/linux-newbie-8/how-to-add-flags-in-libvirt-4175617668/#post5781729

Step 11 - Make libvirt automatically start/stop the md0 disk

  1. Create the file /etc/libvirt/hooks/qemu (assuming the name of the VM is win10)

     1#!/usr/bin/env bash
 2EFIDIR=/media/dataHD/linux/kvm/
 3if [[ $1 == "win10" ]]; then
 4  if [[ $2 == "prepare" ]]; then
 5    echo 3080 > /proc/sys/vm/nr_hugepages
 6    "${EFIDIR}/start_md0"
 7  elif [[ $2 == "release" ]]; then
 8    "${EFIDIR}/stop_md0"
 9    echo 0 > /proc/sys/vm/nr_hugepages
10  fi
11fi

  2. Make it executable

    1sudo chmod u+x /etc/libvirt/hooks/qemu

If everything worked as expected it should now be possible to start the virtual machine from Virt Manager, and it should build the disk and allocate the hugepages memory.

Anki Add-Ons Bulk Update
Automatic Help for Your Shell Aliases