1. devstack
  2. Commit 75afd6d33c93f510fb9f62abd443eac4131480e6
Browse code

guides: Notes to setup DevStack with Nested KVM Virtualization

Add a document with procedure to configure KVM-based nested
virtualization on the physical host and to configure DevStack (in a VM)
to take advantage of it.

Current topics:

- Configure nested virt on Intel hosts
- Configure nested virt on AMD hosts
- Expose virt extensions to DevStack VM
- Ensure DevStack VM is using KVM

Change-Id: Ibe6fa482cc0d51183438d99680a0e10d0da652cb

Kashyap Chamarthy authored on 2015/01/21 01:39:25
Showing 2 changed files
doc/source/guides/devstack-with-nested-kvm.rst
History View file @ 75afd6d
1 1 
new file mode 100644
... ... 
@@ -0,0 +1,139 @@
0 
+=======================================================
1 
+Configure DevStack with KVM-based Nested Virtualization
2 
+=======================================================
3 
+
4 
+When using virtualization technologies like KVM, one can take advantage
5 
+of "Nested VMX" (i.e. the ability to run KVM on KVM) so that the VMs in
6 
+cloud (Nova guests) can run relatively faster than with plain QEMU
7 
+emulation.
8 
+
9 
+Kernels shipped with Linux distributions doesn't have this enabled by
10 
+default. This guide outlines the configuration details to enable nested
11 
+virtualization in KVM-based environments. And how to setup DevStack
12 
+(that'll run in a VM) to take advantage of this.
13 
+
14 
+
15 
+Nested Virtualization Configuration
16 
+===================================
17 
+
18 
+Configure Nested KVM for Intel-based Machines
19 
+---------------------------------------------
20 
+
21 
+Procedure to enable nested KVM virtualization on AMD-based machines.
22 
+
23 
+Check if the nested KVM Kernel parameter is enabled:
24 
+
25 
+::
26 
+
27 
+    cat /sys/module/kvm_intel/parameters/nested
28 
+    N
29 
+
30 
+Temporarily remove the KVM intel Kernel module, enable nested
31 
+virtualization to be persistent across reboots and add the Kernel
32 
+module back:
33 
+
34 
+::
35 
+
36 
+    sudo rmmod kvm-intel
37 
+    sudo sh -c "echo 'options kvm-intel nested=y' >> /etc/modprobe.d/dist.conf"
38 
+    sudo modprobe kvm-intel
39 
+
40 
+Ensure the Nested KVM Kernel module parameter for Intel is enabled on
41 
+the host:
42 
+
43 
+::
44 
+
45 
+    cat /sys/module/kvm_intel/parameters/nested
46 
+    Y
47 
+
48 
+    modinfo kvm_intel | grep nested
49 
+    parm:           nested:bool
50 
+
51 
+Start your VM, now it should have KVM capabilities -- you can verify
52 
+that by ensuring `/dev/kvm` character device is present.
53 
+
54 
+
55 
+Configure Nested KVM for AMD-based Machines
56 
+--------------------------------------------
57 
+
58 
+Procedure to enable nested KVM virtualization on AMD-based machines.
59 
+
60 
+Check if the nested KVM Kernel parameter is enabled:
61 
+
62 
+::
63 
+
64 
+    cat /sys/module/kvm_amd/parameters/nested
65 
+    0
66 
+
67 
+
68 
+Temporarily remove the KVM AMD Kernel module, enable nested
69 
+virtualization to be persistent across reboots and add the Kernel module
70 
+back:
71 
+
72 
+::
73 
+
74 
+    sudo rmmod kvm-amd
75 
+    sudo sh -c "echo 'options amd nested=1' >> /etc/modprobe.d/dist.conf"
76 
+    sudo modprobe kvm-amd
77 
+
78 
+Ensure the Nested KVM Kernel module parameter for AMD is enabled on the
79 
+host:
80 
+
81 
+::
82 
+
83 
+    cat /sys/module/kvm_amd/parameters/nested
84 
+    1
85 
+
86 
+    modinfo kvm_amd | grep -i nested
87 
+    parm:           nested:int
88 
+
89 
+To make the above value persistent across reboots, add an entry in
90 
+/etc/modprobe.ddist.conf so it looks as below::
91 
+
92 
+    cat /etc/modprobe.d/dist.conf
93 
+    options kvm-amd nested=y
94 
+
95 
+
96 
+Expose Virtualization Extensions to DevStack VM
97 
+-----------------------------------------------
98 
+
99 
+Edit the VM's libvirt XML configuration via `virsh` utility:
100 
+
101 
+::
102 
+
103 
+    sudo virsh edit devstack-vm
104 
+
105 
+Add the below snippet to expose the host CPU features to the VM:
106 
+
107 
+::
108 
+
109 
+    <cpu mode='host-passthrough'>
110 
+    </cpu>
111 
+
112 
+
113 
+Ensure DevStack VM is Using KVM
114 
+-------------------------------
115 
+
116 
+Before invoking ``stack.sh`` in the VM, ensure that KVM is enabled. This
117 
+can be verified by checking for the presence of the file `/dev/kvm` in
118 
+your VM. If it is present, DevStack will default to using the config
119 
+attribute `virt_type = kvm` in `/etc/nova.conf`; otherwise, it'll fall
120 
+back to `virt_type=qemu`, i.e. plain QEMU emulation.
121 
+
122 
+Optionally, to explicitly set the type of virtualization, to KVM, by the
123 
+libvirt driver in Nova, the below config attribute can be used in
124 
+DevStack's ``local.conf``:
125 
+
126 
+::
127 
+
128 
+    LIBVIRT_TYPE=kvm
129 
+
130 
+
131 
+Once DevStack is configured succesfully, verify if the Nova instances
132 
+are using KVM by noticing the QEMU CLI invoked by Nova is using the
133 
+parameter `accel=kvm`, e.g.:
134 
+
135 
+::
136 
+
137 
+    ps -ef | grep -i qemu
138 
+    root     29773     1  0 11:24 ?        00:00:00 /usr/bin/qemu-system-x86_64 -machine accel=kvm [. . .]
doc/source/index.rst
History View file @ 75afd6d
... ... 
@@ -66,6 +66,7 @@ Walk through various setups used by stackers
66 66 
    guides/single-machine
67 67 
    guides/multinode-lab
68 68 
    guides/neutron
69 
+   guides/devstack-with-nested-kvm
69 70
70 71 
 All-In-One Single VM
71 72 
 --------------------
... ... 
@@ -94,6 +95,13 @@ Building a DevStack cluster with :doc:`Neutron Networking <guides/neutron>`.
94 94 
 This guide is meant for building lab environments with a dedicated
95 95 
 control node and multiple compute nodes.
96 96
97 
+DevStack with KVM-based Nested Virtualization
98 
+---------------------------------------------
99 
+
100 
+Procedure to setup :doc:`DevStack with KVM-based Nested Virtualization
101 
+<guides/devstack-with-nested-kvm>`. With this setup, Nova instances
102 
+will be more performant than with plain QEMU emulation.
103 
+
97 104 
 DevStack Documentation
98 105 
 ======================
99 106