GPU Acceleration

Note

This document outlines our officially supported GPU integration using the Nvidia container runtime. With a single server deployment it is also possible to use AMD/Intel GPUs on the host by manually configuring Workspace settings. See Manual Intel or AMD GPU configuration for more information.

Kasm Workspaces supports passing GPUs through to container sessions. GPUs can be used for development of machine learning, data science, video encoding, graphics acceleration, and a number of other reasons. All uses of GPUs will require special host configurations. This guide will walk through preparing a bare VM with the default installation of Ubuntu 20.04LTS for GPU acceleration within Kasm Workspaces.

If Kasm is a multi-server deployment, these steps need to be performed on each Agent component only. If you are installing Kasm Workspaces on a single server, these steps would only need to be performed on that single instance.

Caution

The security of multi-tenant container GPU accelation is not well established. This feature should be used with caution and with full understanding of the potential security implications. These features are being provided for by the GPU manufacturer and are merely being orchestrated by Kasm Workspaces. Kasm Technologies Inc provides no warranty of the security of such features provided by GPU vendors. We recommend that GPUs not be shared between different security contexts and/or boundaries.

Installing GPU Driver

Follow the guide bellow for the the appropriate GPU vendor.

In this guide we use an AWS EC2 instance type of g4dn.xlarge with 4 CPUs, 16GB of RAM, and 1 NVIDIA Tesla T4 GPU. Any GPU supported by the NVIDIA Container Toolkit should work. We used a vanilla Ubuntu 20.04LTS AMI, so these steps should not assume anything unique to AWS.

# update OS
sudo apt-get update
sudo apt-get upgrade -y

# install dependencies for nvidia driver
sudo apt-get upgrade -y linux-aws
sudo apt-get install -y gcc make linux-headers-$(uname -r)

# blacklist kernel modules that might interfere
cat << EOF | sudo tee --append /etc/modprobe.d/blacklist.conf
blacklist vga16fb
blacklist nouveau
blacklist rivafb
blacklist nvidiafb
blacklist rivatv
EOF
echo 'GRUB_CMDLINE_LINUX="rdblacklist=nouveau"' | sudo tee --append /etc/default/grub
sudo update-grub

# download latest driver, you may use a different method of obtaining the driver, this method uses the AWS CLI
sudo apt install -y awscli
aws s3 cp --no-sign-request  --recursive s3://ec2-linux-nvidia-drivers/latest/ .
chmod +x NVIDIA-Linux-x86_64*.run

# install
sudo /bin/sh ./NVIDIA-Linux-x86_64*.run -s

# check that card0 and renderD128 exist
ls /dev/dri

# check that nvidia-smi is working, it should report the driver and cuda version and card type
nvidia-smi

Install Kasm Workspaces

Installing Kasm Workspaces automatically installs Docker, if it is not already installed. So this step must be completed prior to installing the GPU container toolkit described in the next step. If you are installing Kasm Workspaces on a single server, follow the standard installation guide. If you are using a multi-server deployment for scalability and redundance, follow the Agent Role installation steps of the Multi Server Guide.

Installing Container Toolkit

To use the GPU(s) inside containers, you need to install the GPU vendor’s container toolkit.

# Install the NVIDIA Container Toolkit
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker

# depending on the configuration of your docker daemon, you may need to restart the Kasm agent container
sudo docker restart kasm_agent

GPU Graphics Acceleration

Graphics acceleration is supported on any Kasm maintained Workspace that bases from the kasmweb/core-ubuntu-focal version 1.11.0 or higher. This includes many single app Workspaces and the full Ubuntu Focal desktop kasmweb/ubuntu-focal-desktop. Building custom Workspaces that have GPU acceleration is also possible if you base from kasmweb/core-ubuntu-focal. In most cases, GPU acceleration will work with all applications, without anything special in the derivative Workspace. In some cases, you may need to launch your application with vglrun. Here is an example of a script that executes Google Chrome with vglrun. The IF statement ensures the GPU has indeed been passed through, and if not launches Chrome normally. This is not necessary for most applications, Chrome happens to be one application that this is necessary for.

if [ -f /opt/VirtualGL/bin/vglrun ] && [ ! -z "\${KASM_EGL_CARD}" ] && [ ! -z "\${KASM_RENDERD}" ] && [ -O "\${KASM_RENDERD}" ] && [ -O "\${KASM_EGL_CARD}" ] ; then
    echo "Starting Chrome with GPU Acceleration on EGL device \${KASM_EGL_CARD}"
    vglrun -d "\${KASM_EGL_CARD}" /opt/google/chrome/google-chrome ${CHROME_ARGS} "\$@"
else
    echo "Starting Chrome"
    /opt/google/chrome/google-chrome ${CHROME_ARGS} "\$@"
fi

GPU acceleration can be verified for the desktop by running glxinfo -B in a terminal.

../_images/glxinfo.png — **Verifying GPU Graphics Acceleration using glxinfo**

GPU accelerated can be verified in Chrome and Chromium based browsers by navigating to chrome://gpu

../_images/chrome_gpu.png — **Verifying GPU Graphics Acceleration in Chrome**

Note

For graphics acceleration, only a single GPU can be used. For non-graphics uses of GPUs, multiple GPUs can be passed through and used by the container. For multi-GPU systems, Kasm automatically load balances the session containers across the available GPUs.

Confirm Kasm GPUs

After installing Workspaces, the GPU driver, and GPU container toolkit; confirm Kasm is picking up the GPUs. Log into Kasm as an administrator, navigate to the Agents view under the Admin section. Here you will see a list of agents in the cluster, since this is a single server installation, there will only be one agent listed. Confirm that the agent shows 1 or more GPUs. If the agent lists 0 GPUs, see the troubleshooting section.

../_images/agents_view.png — **Agents Listing**

../_images/agent_details.png — **Agent GPU Details**

Resource Management

GPUs are treated similar to CPU cores within Kasm Workspaces. Each agent reports the number of CPUs, RAM, and GPUs available to share between containers. By default, a Workspace set to 1 require 1 GPU will mean that an agent with 1 GPU will only be able to support a single session of that workspace. Kasm can allow a single GPU to be shared between multiple containers, this is known as overriding, see the CPU / Memory / GPU Override for more details. Overriding the agent to have 4 GPUs would allow up to 4 containers with the GPU passed through, if each were assigned a single GPU. In the context of graphics acceleration, only a single GPU can be passed through. Only non-graphics applications support multiple GPUs per container to be used.

Modify Workspace

In order for sessions to utilize a GPU, the Workspace must be set to require a GPU. Login as administrator, navigate to Workspaces, in the Workspace list find the target Workspace. Expand the menu by clicking the arrow icon next to the Workspace you want to have a GPU and click Edit. Change the GPU count to the desired number.

../_images/image_resources.webp — **Create ML Desktop session**

Also add the following to the Docker Run Config of the Workspace:

{
  "environment": {
    "NVIDIA_DRIVER_CAPABILITIES": "all"
  }
}

Building ML Desktop (Optional)

Kasm maintains an open source project, Workspaces ML Desktop, a desktop environment intended for developers working on machine learning. This Workspace contains common development tools and frameworks for ML development, but can be used in general for development. The following shows how to download and build the ML Workspace.

git clone https://github.com/kasmtech/workspaces-machine-learning.git
cd workspaces-machine-learning
sudo docker build -t kasmweb/desktop-ml:latest .

Note

If you build images directly on a Kasm Workspaces server, you must disable automatic image pruning in the Agent Settings. While building locally is fine for testing, the build is better done in a CI pipeline and pushed to a central docker registry. Workspaces can be configured to automatically pull image updates from a central Docker registry.

The final step is to define a Workspace that utilizes this. Log into Kasm Workspaces as an administrator and navigate to Workspaces in the Admin panel. Click the Add Workspace button. Fill out the information as shown in the following screenshot. Adjust the cores, memory, and GPU settings as desired. Scroll down and enter the categories Development and Desktop in the Categories field, this ensures the Workspace shows up in those categories on the Workspaces home page.

../_images/image_create.webp — **Create new Workspace**

There are many options and settings that can be applied, such as persistent profiles or two-factor authentication. See our various how-to and configuration guides in this documentation for more details.