Friday, September 30, 2016

Taming VirtualBox: A Few Handy Tips

VirtualBox (referred to as VB hereafter) has become a popular tool for managing virtual machines. While most VM-related tasks in VirtualBox are trivial, there are certain facts, tips ans tricks that can make your life easier.

  • Change your VirtualBox home directory.
  • By default, VirtualBox sets up a VirtualBox VMs directory in your user home. While this can be handy for general usage (for reasons like not requiring a volume mount for accessing your VMs), it is preferable to change the location to a different disk partition (especially if your VMs are expected to persist large amounts of data to disk, such as application traces or Docker image builds) to avoid surprises like "disk full" errors at unexpected (and often critical, as per Murphy's Law) times. In addition, changing the directory name to avoid whitespace can make your life easy, especially on Linux-based systems that generally do not like whitespace in file names or paths.

  • VB eating up your disk space? Shrink your disk images.
  • Many prefer VB's dynamically allocated disk format over the fixed-size one, as it eliminates unnecessarily allocating disk for the full virtual disk capacity while still retaining the potential to grow to that size eventually. However, it still has the downside of not shrinking back despite the deletion of content inside. Hence disk shrinking has to be done manually. Although the process may take a considerable time and may not be entirely friendly for your physical hard disk, it would often yield surprising results.

  • Clean up your VMs regularly.
  • Often the main cause of huge disk images is the accumulation of garbage files, such as logs, traces (e.g. for Oracle DBs), downloaded packages (e.g. /var/cache/apt/archives in systems using apt package manager, /var/lib/docker/tmp containing failed download artifacts in Docker installations) and residue from dev builds (e.g. untagged Docker images). Remember, a dynamically allocated VDI file won't grow unless the already allocated space is fully utilized, hence you reduce the chance of ending up with a huge VDI by doing regular clean-ups as VirtualBox would be able to fit future content into the already allocated but freed-up disk (file) space.

  • Use sparse files for disk image backups.
  • If you back up your workspace with a command like rsync, each 'dirty' virtual machine image file would get copied all over again during every backup cycle, significantly increasing backup time and disk usage while possibly shortening the life of both backup and backed-up media. Moreover, since virtual disks are usually dynamically allocated and hence sparse (consuming just enough space to accompany the current content rather than the entire virtual capacity), backing up the entire disk image file often means an unnecessary transfer of huge zero-valued byte blocks (given that your VM has not grown to its maximum capacity yet, or that you have performed a cleanup but have not yet shrunk your VM to propagate the effect to the physical disk).

    Posts like this and this describe a smarter approach where rsync just copies the changed data (blocks). It basically creates a set of sparse files initially for any files that have not been copied so far, and later runs an in-place update of the files already existing on the backup medium (including the ones created in the aforementioned step) where only the modified blocks get transferred. (Disclaimer: I have not yet validated the speedup gained by this strategy.)

  • Use --type=headless at startup.
  • Often you do not want the started-up VMs to have a visible video console, especially if they merely run some service accessible over the network, or if they offer remote login (e.g. ssh). In such cases you can avoid the cluttering of your workspace (desktop) by starting the VM in headless mode (VBoxManage startvm --type=headless "VM name" in console, or via drop-down next to Start button > Headless start in the UI), where the VM would run in the background without showing any display output window. If you are not sure whether you would need to access the display output later on, you can always go for the detachable start mode (similar to the aforementioned steps) where the VM initially starts on the fronted but can later be sent to the background (by attempting to close the display output and selecting Continue running in the background on the resulting dialog). (The same can be done to hide a headless VM after clicking the Show button while the VM is running.)

  • Disable swapping on your VMs.
  • Swapping usually happens when your VM runs out of memory. With the additional layers of abstraction and overhead of the virtual disk, swapping simply becomes a nightmare over time (probably because, although the VM accesses the swap area in a memory-mapped fashion, the underlying VDI file is part of the regular filesystem and cannot compensate for the required access patterns). In case of Linux-based systems, swapping can easily be disabled at installation time, or by removing swap mount entries from /etc/fstab on an already installed system. Since a VM is usually used for a specific purpose (e.g. running a database server), allocating enough physical memory to be used instead of swap should not be a problem.

For more advanced VB tricks like creating immutable and 'resettable' VMs, visit