May 6, 2026

What I Learned Building a Toy VPS Cloud - Part 1

How does a VM actually get provisioned in something like EC2 or DigitalOcean? I had a rough idea, but the only way to really know was to build a small version myself. The result is Cumulus, a toy VPS cloud running on my laptop. The source is on GitHub.

This is the first post in the series, and it covers the foundation: getting a single VM running with QEMU and configured enough to log in.

The wishlist

I wanted to learn a few specific things — most of these felt like magic the first time I used them, and I wanted to know how each was actually built.

Manage VMs: see what’s running, start and stop them, destroy them when I’m done.
Configure instance sizes: pick how many vCPUs, how much RAM, how big the disk.
Take snapshots: point-in-time backups from a single click, fast.
Choose operating systems: pick from a handful of base images; the new VM comes up already configured on first boot.
SSH into a fresh VM: DigitalOcean or Hostinger boots your machine with your SSH key already installed.
Connect via VNC: open a graphical console of any VM in the browser, no client install.

If I can build all of that, I’ll have a real mental model of how a cloud provider works under the hood, and a feel for the problems they have to solve.

Running VMs

There are several options for running VMs locally. I picked QEMU because it didn’t need much setup and exposed enough internals to let me poke at the parts I cared about. For a real production system I’d probably reach for libvirt which seems to have a more declarative way of handling things.

QEMU

QEMU is the heart of Cumulus, though in theory it could be swapped for any other VM backend. So my first task was to figure out how it worked and how to drive it for what I needed. After some trial and error, I landed on a command that would launch a VM, something like:

qemu-system-aarch64 \
  -accel hvf \
  -machine virt \
  -cpu host \
  -m 2G \
  -drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on \
  -serial unix:serial.sock,server \
  -display none

That’s a lot happening here, what is important to note is the command name. QEMU has different binaries per target architecture (e.g., qemu-system-x86_64). You pick the command matching the architecture you want the guest to be. Let’s discuss each of the params:

-accel hvf: sets the acceleration backend. HVF is the Apple Hypervisor.framework, which is the macOS native virtualization layer, on Linux you would use KVM, Xen, or MSHV. This is important because it tells QEMU to not emulate the CPU in software, meaning that the guest instructions will run in the actual hardware. If we omit this, QEMU falls back to TCG (Tiny Code Generator), which emulates the VM CPU in software and is much slower.
-machine virt: this option is the kind of machine you are emulating. You can run qemu-system-aarch64 -machine help to see the whole list, but the gist of it is that you could choose a Raspberry Pi that QEMU would emulate down to its quirks and limits.
-cpu host: passes the host CPU’s exact model and feature set to the guest, no translation. Since I’m running on a Mac, the guest OS sees an Apple Silicon arm64.
-m 2G: sets the memory size to 2GiB.
-drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on: attaches the UEFI firmware as a read-only raw image through a parallel flash interface.
-serial unix:serial.sock,server: exposes the guest’s serial console as a Unix socket on the host and blocks until a client connects. Inside the guest, that console is a virtual UART — a chip Linux exposes as a TTY (/dev/ttyAMA0) and reads and writes bytes through.
-display none: doesn’t open the framebuffer window on my desktop. The idea is that once Cumulus has more features, we connect to it via SSH or VNC.

Now we can connect to the serial socket from another terminal tab:

socat -,rawer,escape=0x1d unix-connect:serial.sock

On the terminal we see the UEFI shell:

UEFI Interactive Shell v2.2
EDK II
UEFI v2.70 (EDK II, 0x00010000)
map: No mapping found.
Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell>

And this is expected because we don’t have any disks attached. The next step is disks. We could take the same approach as on our own machines: attach an empty volume, grab an OS installer ISO, and run through the install, the way I used to do in the old days. There’s a better way, though — some Linux distros publish cloud images (Ubuntu’s are here) that already have the installation step done for us. We can do something like this:

curl -LO https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-arm64.img

This downloads an Ubuntu cloud image in the qcow2 format. qcow2 is QEMU’s disk image format: instead of representing a raw block device byte-for-byte, it stores the virtual disk in a regular file and allocates space as data is written. It also supports features like snapshots, compression, and backing files, which makes it common for VM and cloud images.

The extension is .img, but if you inspect the file with qemu-img info noble-server-cloudimg-arm64.img, you’ll see that it’s a qcow2 format. Now we repeat the previous command, but we add a new drive from a file:

qemu-system-aarch64 \
    -accel hvf \
    -machine virt \
    -cpu host \
    -m 2G \
    -drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on \
    -drive file=noble-server-cloudimg-arm64.img,if=virtio \
    -serial unix:serial.sock,server \
    -display none

Once we do this and reconnect, we’ll see Ubuntu booting. That’s really fun! After a moment, the login screen appears — and here’s the gotcha. These images don’t ship with a default user and password, so we can’t log in.

How to Log in

The machine we just booted has no information about who’s using it, so we need a way to give it that at boot — enter cloud-init. What’s going to happen: we’ll attach a small volume to the VM with our config inside, and cloud-init will read it on boot and apply it.

Cloud-init is a package that comes pre-installed on those distros and runs at boot time to turn the generic image into a specific configured VM. Cloud-init has several datasources, you can take data from a specific IP or domain, but there is also the NoCloud data source, which, among other things, searches for a file system labeled with cidata with two files: user-data and meta-data, so we need a way to provide those files to the VM at boot time.

As the name implies user-data contains parameters related to the user to be configured, so a minimal example is:

#cloud-config
users:
  - name: cumulus
    plain_text_passwd: cumulus
    lock_passwd: false
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash

There’s also a meta-data file. A minimal example:

instance-id: cumulus-vm-001

We can package those files into a file system and pass it to QEMU with the following command:

mkisofs -output seed.iso -volid cidata -rock user-data meta-data

I used mkisofs to build an ISO 9660 CD-ROM image at seed.iso, with the volume identifier cidata (expected by cloud-init NoCloud) and the two files inside. The -rock flag enables Rock Ridge, which preserves Unix-style filenames; without it, the files would be all uppercase, DOS-era style. Those two files end up at the root of the “CD-ROM”, and we can boot the VM by passing -cdrom seed.iso.

qemu-system-aarch64 \
    -accel hvf \
    -machine virt \
    -cpu host \
    -m 2G \
    -drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on \
    -drive file=noble-server-cloudimg-arm64.img,if=virtio \
    -cdrom seed.iso \
    -serial unix:serial.sock,server \
    -display none

Connect to the serial socket again, wait for it to boot, and we’re in.

What’s Next?

In theory, we could wrap this command into the Cumulus agent and have the Control Plane drive it — and that’s what I ended up doing. But along the way I kept hitting things VPS providers handle that I hadn’t yet thought about: scheduling VMs across hosts, snapshots, picking an operating system, SSH key injection, browser-based VNC consoles. Each gets its own post in the series.

If you’re curious, stay tuned for the rest of the series.