Gaming and VR Virtual Machines

Preamble

The intent of this setup is to provide a base Linux system for Development and native Gaming, while also providing the ability to run Windows-only Games (or Heavy Windows-only software). The setup will consist of 2TB SSD for Linux where a small 200GB Microsoft Windows 10 disk will also reside. The Windows 10 Virtual Machine will have direct access to several devices like SATA and Graphics. In my setup, the VM will have Three SATA SSDs (for Video Game storage) and a PCI-Express based Graphics Card. To facilitate the direct access to the Graphics Card we will be using Virtual Function I/O or VFIO. VFIO along with IOMMU allows VMs to directly access PCI devices (and PCI-Express devices). For this system I am using Arch Linux as the Linux Host Operating System due to its nature of being more bleeding-edge than Fedora. Using a Linux Distribution that is newer helps with patching and fixing problems that will arise with VFIO and PCI Pass-through. (Although, you should be able to apply any patches you might need to any other Linux distribution.) We are going to be building a Microsoft Windows 10 VM with type Q35, firmware OVMF, and with a qcow2 Primary (C:/) SATA disk.

Hardware Example

  • AMD Ryzen 9 3900X 12c/24t 3.8GHz CPU

  • 64GB DDR4-3600 Dual Channel CL16 4x16GB RAM

  • ASUS Crosshair VIII Hero (Non-WiFi Edition, UEFI BIOS ver 1302)

  • AMD Sapphire Nitro+ RX 5700 XT 8GB Video Card (Slot 2, Win10 VM Guest)

  • AMD Sapphire Pulse RX Vega 56 8GB Video Card (Slot 1, Linux VM Host)

  • Samsung 970 EVO Plus 2TB (Linux Host Storage)

  • 3x Samsung 860 QVO 1TB SSDs (Win10 VM Games Storage, SATA Passthrough)

  • PCI-Express x4 USB3 Controller (Win10 VM)

The Linux Host GPU (Vega 56) will be in slot PCIE16_1 and the Win10 Guest GPU (5700 XT) will be in slot PCIE16_2. Review your motherboard’s manual for PCI-Express Bandwidth limitations, and place your GPUs appropriately to ensure the performance doesn’t suffer.

BIOS Setup

You might need to enable the following in your bios if SVM is disabled by default.

Ensure AMD SVM is Enabled by navigating to 'Advanced\CPU Configuration` and down to SVM Mode. Toggle to Enabled.

You might need to enable the following in your bios settings, but things should work by default or on auto.

Ensure AMD IOMMU is Enabled by navigating to 'Advanced\AMD CBS\NBIO Common Options` and down to IOMMU. Toggle to Enabled. Ensure ACS is Enabled by navigating to 'Advanced\AMD CBS\NBIO Common Options` and down to ACS Enable. Toggle to Enable.

Linux Kernel Patching (Navi 10 Patch)

The Navi 10 Patch (or Vega 56 Patch) is required, and if not applied, will induce instability or system hanging of the Linux Host.

The first problem I ran into with my hardware is that the 5700XT is based on the Navi 10 architecture. Navi 10 support in the linux kernel is limited to the amdgpu kernel driver, which in our case will be replaced with the vfio-pci driver. The problem is encountered by attempting to reboot or hard reset the VM, and the GPU/PCI device isn’t properly reset and reinitialized. This functionality is called Function Level Reset or PCI Device Reset. The amdgpu driver in your kernel version may or may not have a patch/fix for Function Level Reset routines, but it doesn’t help anyways in our case. Luckly, we have a patch by 'gnif' that adds a PCI Device Reset to the PCI quirks driver.

Libvirt, Passthrough, and Other Special Configuration

Information on Configuring Linux Drivers and VFIO is outlined in Appendix A.

The following sections are the additions or modifications required to optimize the VM for VR and Gaming.

CPU

The following code section details the pinning of vCPUs to physical CPU Cores. The VM is emulating a 6c/12t and pinning those vCPUs to logical CPUs 2-6 and 14-19 (where the 3900X has 24 total logical CPUs and where Logical CPUs 0 and 12 are mapped to Core 0) will help with performance of the VM.

  <vcpu placement='static' cpuset='2-7,14-19'>12</vcpu>
  <iothreads>1</iothreads>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='14'/>
    <vcpupin vcpu='2' cpuset='3'/>
    <vcpupin vcpu='3' cpuset='15'/>
    <vcpupin vcpu='4' cpuset='4'/>
    <vcpupin vcpu='5' cpuset='16'/>
    <vcpupin vcpu='6' cpuset='5'/>
    <vcpupin vcpu='7' cpuset='17'/>
    <vcpupin vcpu='8' cpuset='6'/>
    <vcpupin vcpu='9' cpuset='18'/>
    <vcpupin vcpu='10' cpuset='7'/>
    <vcpupin vcpu='11' cpuset='19'/>
    <emulatorpin cpuset='0,12'/>
    <iothreadpin iothread='1' cpuset='0,12'/>
  </cputune>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' dies='1' cores='6' threads='2'/>
    <feature policy='require' name='topoext'/>
    <feature policy='disable' name='amd-stibp'/>
  </cpu>

GPU and HDMI Audio Device

To enable the use of your GPU for hardware acceleration you must find the PCI address for your GPU and its PCI functions. For the 5700XT, you should find the Main GPU address (function 0x00) and a HDMI Audio device (function 0x01). Newer nVidia cards will have two more functions.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x0f' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x0f' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
    </hostdev>

SATA (Pass-through)

For storage of my games I used three SATA SSDs in a Windows Storage Setup (RAID 0). To emulate the same functionallity I passed through all three devices into KVM.

<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='writeback' io='threads' discard='unmap'/>
  <source dev='/dev/disk/by-id/ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706444T' index='3'/>
  <target dev='sda' bus='scsi'/>
</disk>
<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='writeback' io='threads' discard='unmap'/>
  <source dev='/dev/disk/by-id/ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706466F' index='2'/>
  <target dev='sdb' bus='scsi'/>
</disk>
<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='writeback' io='threads' discard='unmap'/>
  <source dev='/dev/disk/by-id/ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706471J' index='1'/>
  <target dev='sdc' bus='scsi'/>
</disk>
<controller type='scsi' index='0' model='virtio-scsi'>
  <driver queues='8' iothread='1'/>
</controller>

USB 3 Controller (Pass-through for Oculus Rift)

To enable the use of my Oculus Rift, I had to pass in a full USB 3.0 Controller to KVM.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/>
    </hostdev>

Setup Looking Glass

For the most part one can follow the official Installation instructions: https://looking-glass.hostfission.com/wiki/Installation

Linux Changes

/etc/tmpfiles.d/10-looking-glass.conf:
f /dev/shm/looking-glass 0660 user kvm -

Replace <user> with your username.

Ask systemd-tmpfiles to create the shared memory file now without waiting to next boot

systemd-tmpfiles --create /etc/tmpfiles.d/10-looking-glass.conf

Build looking-glass-client Follow instructions from the Looking Glass website.

Modify Virtual Machine Add the following before the closing domain xml:

<shmem name='looking-glass'>
    <model type='ivshmem-plain'/>
    <size unit='M'>64</size>
    <alias name='shmem0'/>
    <address type='pci' domain='0x0000' bus='0x08' slot='0x01' function='0x0'/>
</shmem>

Install 'looking-glass-host' on Windows 10 Download the Bleeding-Edge releases if the client is built from the master branch

Appendix

Appendix A: VFIO & Linux Configuration

/etc/default/grub:
GRUB_CMDLINE_LINUX_DEFAULT="... mce=off video=efifb:off iommu=1 amd_iommu=on avic=1 rd.driver.pre=vfio-pci vfio-pci.ids=1002:731f,1002:ab38,1912:0014 pcie_acs_override=downstream ..."
/etc/modprobe.d/kvm.conf:
options kvm ignore_msrs=1 report_ignored_msrs=0
options kvm_amd avic=1
/etc/modprobe.d/vfio.conf:
softdep radeon pre: vfio-pci
softdep amdgpu pre: vfio-pci
softdep nouveau pre: vfio-pci
softdep drm pre: vfio-pci

Appendix B: Linux Kernel Patching

There’s a great blog post on patching RHEL kernel patching by @Vincent Cojot:

https://mojo.redhat.com/community/communities-at-red-hat/infrastructure/cloud-platforms-community-of-practice/blog/2019/09/19/when-you-need-that-kernel-hotfix-now

For Arch Linux, use the following wiki page:

https://wiki.archlinux.org/index.php/Kernel/Arch_Build_System

Appendix C: IOMMU Groups and Device Selection

IOMMU Groups:

#!/bin/bash
shopt -s nullglob
for g in /sys/kernel/iommu_groups/*; do
  echo "IOMMU Group ${g##*/}:"
  for d in $g/devices/*; do
    echo -e "\t$(lspci -nns ${d##*/})"
  done
  echo ""
done

Example:

IOMMU Group 0:
     00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]

IOMMU Group 1:
     00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]

IOMMU Group 10:
     00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]

IOMMU Group 11:
     00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]

IOMMU Group 12:
     00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]

IOMMU Group 13:
     00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]

IOMMU Group 14:
     00:08.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]

IOMMU Group 15:
     00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
     00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)

IOMMU Group 16:
     00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1440]
     00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1441]
     00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1442]
     00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1443]
     00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1444]
     00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1445]
     00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1446]
     00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 [1022:1447]

IOMMU Group 17:
     01:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]

IOMMU Group 18:
     02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream [1022:57ad]

IOMMU Group 19:
     03:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]

IOMMU Group 2:
     00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]

IOMMU Group 20:
     03:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]

IOMMU Group 21:
     03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]

IOMMU Group 22:
     03:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]

IOMMU Group 23:
     03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]

IOMMU Group 24:
     03:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]

IOMMU Group 25:
     03:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]

IOMMU Group 26:
     04:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1470] (rev c3)

IOMMU Group 27:
     05:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1471]

IOMMU Group 28:
     06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XL/XT [Radeon RX Vega 56/64] [1002:687f] (rev c3)

IOMMU Group 29:
     06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]

IOMMU Group 3:
     00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]

IOMMU Group 30:
     07:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125]

IOMMU Group 31:
     08:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03)

IOMMU Group 32:
     09:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)

IOMMU Group 33:
     0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
     0a:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
     0a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]

IOMMU Group 34:
     0b:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)

IOMMU Group 35:
     0c:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)

IOMMU Group 36:
     0d:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev c1)

IOMMU Group 37:
     0e:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479]

IOMMU Group 38:
     0f:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev c1)

IOMMU Group 39:
     0f:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio [1002:ab38]

IOMMU Group 4:
     00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]

IOMMU Group 40:
     10:00.0 Ethernet controller [0200]: Mellanox Technologies MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] [15b3:6750] (rev b0)

IOMMU Group 41:
     11:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]

IOMMU Group 42:
     12:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]

IOMMU Group 43:
     12:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]

IOMMU Group 44:
     12:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]

IOMMU Group 45:
     13:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)

IOMMU Group 46:
     14:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)

IOMMU Group 5:
     00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]

IOMMU Group 6:
     00:03.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]

IOMMU Group 7:
     00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]

IOMMU Group 8:
     00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]

IOMMU Group 9:
     00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]

USB

To find out the USB Controller to PCI Bus mapping, execute:

$ sudo lsusb -v 2>/dev/null | grep '^Bus\|iSerial'

Example:

[snelson@hkn-desktop ~]$ sudo lsusb -v 2>/dev/null | grep '^Bus\|iSerial'
Bus 006 Device 004: ID 2109:0812 VIA Labs, Inc. VL812 Hub
  iSerial                 0
Bus 006 Device 003: ID 2109:8110 VIA Labs, Inc. Hub
  iSerial                 0
Bus 006 Device 002: ID 2109:0812 VIA Labs, Inc. VL812 Hub
  iSerial                 0
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
  iSerial                 1 0000:11:00.3
Bus 005 Device 012: ID 046d:0ab9 Logitech, Inc. USB2.0 Hub
  iSerial                 3 2009BAA00C58
Bus 005 Device 010: ID 1038:12ad SteelSeries ApS
  iSerial                 0
Bus 005 Device 007: ID 2109:2812 VIA Labs, Inc. VL812 Hub
  iSerial                 0
Bus 005 Device 011: ID 0fd9:0060 Elgato Systems GmbH
  iSerial                 3 AL19H1A02034
Bus 005 Device 005: ID 2109:2811 VIA Labs, Inc. Hub
  iSerial                 0
Bus 005 Device 009: ID 1b1c:1b66 Corsair
  iSerial                 3 32759A0C0169495F
Bus 005 Device 006: ID 1b1c:1b5d Corsair
  iSerial                 3 1400C024AF2609095A3F353AF5001BC0
Bus 005 Device 004: ID 2109:2813 VIA Labs, Inc. USB2.0 Hub
  iSerial                 0
Bus 005 Device 003: ID 1b1c:1b49 Corsair
  iSerial                 3 0201402EAF1E90C35A6DD6DBF5001BC2
Bus 005 Device 002: ID 2109:2812 VIA Labs, Inc. VL812 Hub
  iSerial                 0
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  iSerial                 1 0000:11:00.3
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
  iSerial                 1 0000:07:00.3
Bus 003 Device 002: ID 04e6:5116 SCM Microsystems, Inc. SCR331-LC1 / SCR3310 SmartCard Reader
  iSerial                 5 53311916202490
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  iSerial                 1 0000:07:00.3
Bus 002 Device 004: ID 2833:0211 Oculus VR Rift Sensor
  iSerial                 3 WMTD303A200X4N
Bus 002 Device 003: ID 2833:3031 Oculus VR, Inc. Rift
  iSerial                 3 123456789A
Bus 002 Device 002: ID 2833:0211 Oculus VR Rift Sensor
  iSerial                 3 WMTD303Q602WUN
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
  iSerial                 1 0000:07:00.1
Bus 001 Device 009: ID 0b05:18f3 ASUSTek Computer, Inc.
  iSerial                 3 9876543210
Bus 001 Device 007: ID 1b1c:1d00 Corsair
  iSerial               128 030501A9941C1D1C
Bus 001 Device 010: ID 1b1c:0c21 Corsair
  iSerial                 0
Bus 001 Device 008: ID 1b1c:0c1a Corsair
  iSerial                 3 2AC0000DC92046AFBF8F195CC11B00F5
Bus 001 Device 006: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
  iSerial                 0
Bus 001 Device 004: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
  iSerial                 0
Bus 001 Device 005: ID 2833:0330
  iSerial                 3 WMHD303A300M4
Bus 001 Device 003: ID 2833:0031 Oculus VR, Inc. Rift
  iSerial                 3 WMHD303A300M4C
Bus 001 Device 002: ID 2833:2031 Oculus VR, Inc. Rift
  iSerial                 3 123456789A
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  iSerial                 1 0000:07:00.1

SATA

To find out the location of your Physical disk, execute:

$ ls -l /dev/disk/by-id

Example:

[snelson@hkn-desktop ~]$ ls -l /dev/disk/by-id
total 0
lrwxrwxrwx 1 root root  9 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706444T -> ../../sda
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706444T-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706444T-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706444T-part3 -> ../../sda3
lrwxrwxrwx 1 root root  9 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706466F -> ../../sdc
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706466F-part1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706466F-part2 -> ../../sdc2
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706466F-part3 -> ../../sdc3
lrwxrwxrwx 1 root root  9 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706471J -> ../../sdb
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706471J-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706471J-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 May 28 14:08 ata-Samsung_SSD_860_QVO_1TB_S4PGNF0M706471J-part3 -> ../../sdb3
lrwxrwxrwx 1 root root 13 May 28 12:36 nvme-Samsung_SSD_970_EVO_Plus_2TB_S59CNJ0N301646B -> ../../nvme0n1
lrwxrwxrwx 1 root root 15 May 28 12:36 nvme-Samsung_SSD_970_EVO_Plus_2TB_S59CNJ0N301646B-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 15 May 28 12:36 nvme-Samsung_SSD_970_EVO_Plus_2TB_S59CNJ0N301646B-part3 -> ../../nvme0n1p3
lrwxrwxrwx 1 root root 15 May 28 12:36 nvme-Samsung_SSD_970_EVO_Plus_2TB_S59CNJ0N301646B-part4 -> ../../nvme0n1p4

Appendix D: libvirt patch for AMD-STIBP bug

Add the following after the 'ibpb' and before 'amd-ssbd' features in '/usr/share/libvirt/cpu_map/x86_features.xml':

  <feature name='amd-stibp'>
    <cpuid eax_in='0x80000008' ebx='0x00008000'/>
  </feature>

Afterwards you can add:

  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' dies='1' cores='6' threads='2'/>
    <feature policy='require' name='topoext'/>
    <feature policy='disable' name='amd-stibp'/><!-- add this -->
  </cpu>

Appendix E: Full LibVirt XML

Full LibVirt Win10 XML

References

https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF

https://forum.level1techs.com/t/navi-reset-kernel-patch/147547

https://www.reddit.com/r/VFIO/comments/gs33gy/getting_hostpassthrough_working_with_libvirt_630/

https://www.tauceti.blog/post/linux-amd-x570-nvidia-gpu-pci-passthrough-1-the-hardware/

https://www.tauceti.blog/post/linux-amd-x570-nvidia-gpu-pci-passthrough-5-looking-glass/

https://heiko-sieger.info/running-windows-10-on-linux-using-kvm-with-vga-passthrough/

https://www.reddit.com/r/VFIO/