Discovering Kubevirt..

James Tate

• 6 min read
Discovering Kubevirt..

I recently saw that kubevirt has reached version 1 and wanted to try it out. Kubevirt is basically running traditional virtual machines inside containers and can be incredibly useful for VDI type implementations, dynamic provisioning and auto-scaling environments as the programmatic control afforded by a Kubernetes environment is very powerful in this regard.

So lets start with the basis of the install….you’re going to need a working cluster, so for arguments sake use the streamlined k3s install I’ve built previously. Follow Part 4 and get the cluster installed with Kasten:

Kasten K10 Guide for beginners – Part 3

Next up were going to install the following components:

  1. Kubevirt Operator and CR
  2. Krew with virt plugin
  3. Pull virtio drivers image
  4. Kubevirt manager
  5. Virtctl binary

I’ve written a script that will do all of that for you:

curl -s https://raw.githubusercontent.com/jdtate101/jdtate101/main/kubevirt.sh | bash

Once we’ve done that there are a couple more things to do in order to start the installation of a guest OS. Firstly we need to expose the Kubevirt manager, you can either expose this via an ingress rule or on the loadbalancer, depending upon your preference, but assuming you did this you will end up with the following console:

We can use this to get access to the VM console to configure the VM, otherwise you have to muck about with kubectl remote access, install a console VNC client and configure x-forwarding…managing it through a WebUI is much easier…trust me!!

Next up we need to install a guest VM. In order to do this we first need to create a namespace for it and upload a cd ISO image to it, done by creating a PVC and coping the iso up. I’m going to use Windows 10 as an example in this case. In order to upload we need to send the iso image to the cdi upload proxy (installed as part of the operator). If you list out the services (kubectl get svc -A) you will see under the cdi namespace a service called cdi-uploadproxy….make a note of the clusterIP address, we will need that in a second.

Let’s say we created a new namespace called win10 (kubectl create ns win10) for our VM, we can upload the ISO to it thus:

kubectl virt image-upload --image-path ./Win10_2105_English_x64.iso --pvc-name win10cd-pvc --access-mode ReadWriteOnce --pvc-size 5G --uploadproxy-url https://<cdi-uploadproxy-cluster-ipaddress> --insecure --wait-secs=240 -n win10

Just make sure you set the pvc size to be slightly larger than your ISO image size. This will create the PVC in your namespace, that we will use to bootstrap the VM later.

Next we need to create a manifest for the VM, like this for our windows 10:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: win10hd-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: win10
spec:
  running: false
  template:
    metadata:
      labels:
        kubevirt.io/domain: win10
    spec:
      domain:
        cpu:
          cores: 6
        devices:
          disks:
          - bootOrder: 1
            cdrom:
              bus: sata
            name: cdromiso
          - bootOrder: 2
            disk:
              bus: virtio
            name: harddrive
          - bootOrder: 3
            cdrom:
              bus: sata
            name: virtiocontainerdisk
        machine:
          type: q35
        resources:
          requests:
            memory: 8G
      volumes:
      - name: cdromiso
        persistentVolumeClaim:
          claimName: win10cd-pvc
      - name: harddrive
        persistentVolumeClaim:
          claimName: win10hd-pvc
      - containerDisk:
          image: quay.io/kubevirt/virtio-container-disk
        name: virtiocontainerdisk

You can customise the number of CPU cores, memory and OS disk. You will need a very beefy computer to run this on, I recommend specifying the storage class if you can and using NVMe if at all possible. Note we have referenced the PVC for the ISO image as a cdrom boot device (boot order). We then need to apply it:

kubectl apply -f https://raw.githubusercontent.com/jdtate101/jdtate101/main/win10-kubevirt.yaml -n win10

The VM is not started currently as the manifest had a running: false flag set, we will need to manually start it:

kubectl virt start win10 -n win10

Next check it’s up and running ok:

kubectl get vmi -A

Once the PHASE state is RUNNING, you can move over to the Kubevirt Manager and browse to Virtual machines on the left menu, you should see your running VM. Under the ACTIONS part you can get access to the console to perform the install.

One thing to note is that Windows 10 does NOT include the required virtio drivers for disk access, so during install it will find zero hard disks to install the OS. This is true for all windows OS installs, apart from windows 11 I believe, I’ve not tried that yet so I cannot confirm. Linux installs should work on as they include the drivers by default. I’m not going to cover it as there are plenty of guides on YouTube on how to load the drivers in…one such on is here (skip ahead to the OS install part around 7mins40s). All the VirtIO drivers are represented as a virtual CD available to you in the OS:

Once you’ve booted you will see under device manager there are a few non-functional components (Network, SCSI drivers etc)…just load the drivers from the virtIo cd image and you should be good to go. You can optionally expose the RDP port for direct access, this can be done manually or use the virtctl binary I’ve loaded in from the script: 

virtctl expose vmi win10 -n win10 --type LoadBalancer --name win10rdp --port=3389

virtctl just avoids mucking about with the dynamic pod names, as every time you start or stop the VM, the pod name will change, thus it will break your expose.

And that should be all you need to do….. It should be noted that whilst you can backup and restore these VM’s inside kubevirt using Kasten K10, it’s only offlicially supported fully on RedHat’s OpenShift platform, all other uses, such as this implementation…you are on your own. That may change as the technology matures and reaches mass adoption, but for now…please do NOT run kubevirt in production on unsupported platforms if you require Kasten data protection!!!

The kubevirt script is below for those who want to see it:

#! /bin/bash
echo "Starting setup of Kubevirt environment.."
echo ""
sleep 1
export VERSION=$(curl -s https://api.github.com/repos/kubevirt/kubevirt/releases | grep tag_name | grep -v -- '-rc' | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
kubectl create -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml
kubectl create -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml
echo ""
echo "Installing Krew and virt plugin..."
echo ""
sleep 1
(  set -x; cd "$(mktemp -d)" &&  OS="$(uname | tr '[:upper:]' '[:lower:]')" &&  ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&  KREW="krew-${OS}_${ARCH}" &&  curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&  tar zxvf "${KREW}.tar.gz" &&  ./"${KREW}" install krew)
export PATH="${PATH}:${HOME}/.krew/bin"
echo "export PATH="${PATH}:${HOME}/.krew/bin"" >> /root/.bashrc
kubectl krew install virt
export CDIVERSION=$(curl -s https://api.github.com/repos/kubevirt/containerized-data-importer/releases | grep tag_name | grep -v -- '-rc' | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$CDIVERSION/cdi-operator.yaml
kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$CDIVERSION/cdi-cr.yaml
echo ""
echo "Installing Virtio Drivers.."
echo ""
sleep 1
ctr image pull docker.io/kubevirt/virtio-container-disk:latest -k
echo  ""
echo "Finished, installing Kubevirt Manager....."
echo ""
sleep 1
kubectl apply -f https://raw.githubusercontent.com/kubevirt-manager/kubevirt-manager/main/kubernetes/bundled.yaml
echo ""
sleep 1
echo "Install Virtctl..."
echo ""
(  set -x; cd "$(mktemp -d)" &&  OS="$(uname | tr '[:upper:]' '[:lower:]')" &&  ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" && VIRTCTL="virtctl-v${VERSION}-${OS}-${ARCH}" && wget "https://github.com/kubevirt/kubevirt/releases/download/v${VERSION}/${VIRTCTL}" && mv virtctl* /usr/local/bin/virtctl && chmod +x /usr/local/bin/virtctl )
echo ""
exit