Shrink, optimise and expand an existing QCOW2 image

A virtual disk image is a block device in a file. There are a number of different disk image formats to choose from when setting up a virtual machine. QEMU Copy On Write version 2 (QCOW2) is the default virtual disk image format for the Quick Emulator (QEMU). Features such as thin provisioning, snapshots and compression make QCOW2 one of the most versatile virtual disk formats available.

These instructions specifically target Debian 12 with a GNOME desktop as the host, but they should also be applicable to other Linux distributions such as Ubuntu or Linux Mint. The guest in this particular example is a Windows 11 virtual machine that has run out of space.

The overall objective is to shrink and optimise the 64 GiB disk image for random read and write operations before expanding it to a desired size of 128 GiB.

With thanks to Fam Zheng.

Before you begin

Shut down the virtual machine and delete all existing snapshots from the image file.

Never modify images currently in use by a running virtual machine.

Step 1

On the host, install the necessary tools for working with virtual disk images.

$ sudo apt-get install --yes libguestfs-tools gnome-disk-utility

Step 2

Only root can access the host directory /var/lib/libvirt/images. Use the following command to obtain the necessary privileges.

$ sudo su

Step 3

Continue by creating a directory in which to keep your virtual machine backups.

# mkdir /var/lib/libvirt/backups

Step 4

Now create a backup of the virtual machine with the name windows by copying its QCOW2 image file to the backups directory.

# cp /var/lib/libvirt/images/windows.qcow2 /var/lib/libvirt/backups/windows-backup.qcow2

Step 5

Sparsify the image file to convert any free space within the disk image to free space on the host.

# virt-sparsify --in-place /var/lib/libvirt/images/windows.qcow2

Step 6

Rename the sparsified image file.

# mv /var/lib/libvirt/images/windows.qcow2 /var/lib/libvirt/images/windows-sparsified.qcow2

Step 7

Check the disk size of the sparsified image file.

# qemu-img info /var/lib/libvirt/images/windows-sparsified.qcow2

The disk size should be smaller than the virtual size. In this particular case, the disk size is 33.7 GiB and the virtual size 64 GiB.

image: /var/lib/libvirt/images/windows-sparsified.qcow2
file format: qcow2
virtual size: 64 GiB (68719476736 bytes)
disk size: 33.7 GiB
cluster_size: 65536
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: true
refcount bits: 16
corrupt: false
extended l2: false

Step 8

Determine which partition to resize by obtaining more detailed information about the contents of the sparsified disk image.

# virt-filesystems --long -h --all -a /var/lib/libvirt/images/windows-sparsified.qcow2

On the virtual device /dev/sda, the partition /dev/sda3 is equivalent to the Local Disk (C:) of the Windows 11 virtual machine.

Name       Type        VFS   Label  MBR  Size  Parent
/dev/sda1 filesystem vfat - - 96M -
/dev/sda3 filesystem ntfs - - 63G -
/dev/sda4 filesystem ntfs - - 768M -
/dev/sda1 partition - - - 100M /dev/sda
/dev/sda2 partition - - - 16M /dev/sda
/dev/sda3 partition - - - 63G /dev/sda
/dev/sda4 partition - - - 768M /dev/sda
/dev/sda device - - - 64G -

Step 9

Load the network block device (NBD) kernel module.

# modprobe nbd max_part=8

Step 10

Connect the sparsified image.

# qemu-nbd --connect=/dev/nbd9 /var/lib/libvirt/images/windows-sparsified.qcow2

Step 11

The partition /dev/sda3 listed in Step 8 is equivalent to /dev/nbd9p3 connected as a network block device. Use GNOME Disks to shrink /dev/nbd9p3 to its Minimal Size.

Use a graphical utility to minimise the risk of introducing errors.

This image has an empty alt attribute; its file name is disks-options-825x586.png
Select the correct partition and from the pop-up menu, choose the option Resize…
This image has an empty alt attribute; its file name is resize-volume.png
Select Minimal Size and resize the partition.

Step 12

Disconnect the resized image.

# qemu-nbd -d /dev/nbd9

Step 13

Unload the NBD kernel module.

# modprobe -r nbd

Step 14

Create a target image larger than the resized source image. In this example, the size of the target image is 128G and its format QCOW2 with full preallocation and a cluster size of 2M.

# qemu-img create -f qcow2 -o preallocation=full -o cluster_size=2M /var/lib/libvirt/images/windows-target.qcow2 128G

Step 15

Copy the source image to the target image. Specify the correct partition which to expand in the process.

# virt-resize --expand /dev/sda3 /var/lib/libvirt/images/windows-sparsified.qcow2 /var/lib/libvirt/images/windows-target.qcow2

Step 16

Confirm the size of the target image.

# qemu-img info /var/lib/libvirt/images/windows-target.qcow2

The overall disk size is now 128 GiB in total.

image: /var/lib/libvirt/images/windows-target.qcow2
file format: qcow2
virtual size: 128 GiB (137438953472 bytes)
disk size: 128 GiB
cluster_size: 2097152
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: false
refcount bits: 16
corrupt: false
extended l2: false

Step 17

Obtain more detailed information about the contents of the target disk image.

# virt-filesystems --long -h --all -a /var/lib/libvirt/images/windows-target.qcow2

The partition /dev/sda3 of the virtual device /dev/sda is now 127G in size.

Name       Type        VFS   Label  MBR  Size  Parent
/dev/sda1 filesystem vfat - - 96M -
/dev/sda3 filesystem ntfs - - 127G -
/dev/sda4 filesystem ntfs - - 768M -
/dev/sda1 partition - - - 100M /dev/sda
/dev/sda2 partition - - - 16M /dev/sda
/dev/sda3 partition - - - 127G /dev/sda
/dev/sda4 partition - - - 768M /dev/sda
/dev/sda device - - - 128G -

Step 18

Rename the target image file.

# mv /var/lib/libvirt/images/windows-target.qcow2 /var/lib/libvirt/images/windows.qcow2

All done!

You can also modify format specific options for an existing image without having to create a target disk image. Or alternatively expand into a target image that uses a format compatible with other hypervisors, such as RAW, VMDK, VDI, VHD, VHDX or QED.

Can passkeys replace passwords

“Passkeys and the WebAuthn specification were intended to make public key cryptography accessible to average users, rather than just the domain of the tech-savvy. If done right, they could seriously improve security on the Web.” @Drbruced summarises why passkeys are such a good idea in theory and explains where current implementations of the technology fall down in practice.

systemsapproach.org

Was Griechenland wusste: das tödliche Geschäft des Abu Sultan

“Eine monatelange Untersuchung unter der Leitung des griechischen Investigativmediums Solomon und des Netzwerks Arab Reporters for Investigative Journalism in Zusammenarbeit mit El País und der taz ergab, dass die griechischen Behörden schon früh wussten, dass die ‘Pylos 9’ unschuldig waren, sie aber fast ein ganzes Jahr in Untersuchungshaft behielten.” Griechenland kriminalisiert systematisch die Opfer eines der schwersten Schiffsunglücke im Mittelmeer, um von den Versäumnissen der eigenen Behörden abzulenken.

taz.de

Viele gute Gründe für das Fediverse

“Die vor uns stehenden Umwälzungen der digitalen Transformation sind von historischer Dimension. Sie sind in ihrer Bedeutung vergleichbar mit den Veränderungen im Zeitalter der Aufklärung, die die Grundlage für die Menschenrechte und ein friedlich vereintes Europa legten.”

Mario Birkholz sieht die Hochschulen in der Pflicht, ihre digitale Kommunikation mit den Anfoderungen des demokratischen Gemeinwesens in Einklang zu bringen.

netzpolitik.org

How to install Espanso on Debian 12 bookworm from source

Unmet dependencies continue to prevent available Espanso packages from installing on my machine running Debian 12. Given that I depend on Espanso to expand all kinds of text and wanted to upgrade to the latest version for Wayland, I updated the following instructions and installed from source.

Following these instructions, Espanso 2.2.1 for Wayland will be installed on your system and enabled for the current user.

Compiling Espanso from source code

Side-step any dependency problems by compliling Espanso from source and moving the binary into place.

Step 1

Install the required C/C++ compiler and some additional tools.

$ sudo apt-get install --yes build-essential curl git wl-clipboard libxkbcommon-dev libdbus-1-dev libwxgtk3.*-dev libssl-dev

Step 2

Install the required Rust compiler, which is managed by the rustup tool.

$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh&&source ~/.bashrc

Press [Enter] to proceed with the installation.

Current installation options:

1) Proceed with instalation (default)
2) Customize installation
3) Cancel installation
>

Install cargo-make, which is required during the build process.

$ cargo install --force cargo-make --version 0.34.0

Step 3

Get the source code by cloning the Espanso repository to the local directory ~/.local/src/espanso.

$ git clone --progress https://github.com/federico-terzi/espanso ~/.local/src/espanso

Step 4

Compile the Espanso binary as a Wayland-only build in release mode.

$ cargo make --cwd ~/.local/src/espanso --profile release --env NO_X11=true build-binary

Move the resulting binary to the /usr/local/bin directory.

$ sudo mv ~/.local/src/espanso/target/release/espanso /usr/local/bin/

Step 5

Give Espanso the permissions required for operation.

$ sudo setcap "cap_dac_override+p" $(which espanso)

Check to see if the Espanso binary was installed successfuly.

$ espanso --version

Step 6

Register Espanso as a systemd service.

$ espanso service register

Launch Espanso.

$ espanso start && espanso status

Step 7

GNOME desktop only: remove the conflicting default shortcut for activating the window menu.

Settings > Keyboard > Keyboard Shortcuts > View and Customize Shortcuts > Windows > Activate the window menu > [Backspace]

Use [Alt + Space] to open Espanso’s Search bar.

All Done!

Experimental support for Wayland

Espanso has some known limitations under Wayland. Most notably, “there is currently no support for App-specific configurations”.

Big ‘Thank you’ to Federico Terzi for creating Espanso!

Use the Zyklop Pocket for bicycle maintenance

These days, the Wera Zyklop Pocket ratchet is my go-to tool for any kind of bicycle-related maintenance. It is easily small enough to fit in my hip pack and I carry it everywhere I ride. Get a few additonal bits to fit the particular requirements of your bike and you’ve got a fantastic tool that, because of its ergonomic properties, represents a significant step up on any of the traditional multi-tools.

www.wera.de

With regards to Tomas.

A hacker’s perspective: social media account takeover prevention guide

“If you watched the SEC Twitter account hack that moved markets yesterday and wondered how to prevent account takeover for your personal, business, or high profile social media account, here’s an Account Takeover Prevention Guide for you and/or your organization.” @racheltobac neatly summarises the steps you should take to prevent the hijacking of your online accounts.

www.linkedin.com

Monitoring storage devices with smartmontools on Debian or Ubuntu

The acronym SMART stands for Self-Monitoring, Analysis and Reporting Technology and is a monitoring system built into most modern storage devices. The package smartmontools includes the utilities smartctl and smartd, which process SMART data to ‘provide advanced warning of disk degradation and failure‘.

Step 1

Start by configuring nullmailer to receive status updates from your system.

Step 2

Install smartmontools and update its drive database to the latest version.

$ sudo apt-get install --yes smartmontools smart-notifier && sudo update-smart-drivedb

Step 3

Continue by obtaining relevant information about available storage devices.

$ sudo smartctl --scan

Depending on the type of disk, you should see a block of information similar to the following.

/dev/sda -d scsi # /dev/sda, SCSI device

Step 4

Enable SMART support for and display detailed information about the device.

$ sudo smartctl -iHs on /dev/sda

Ideally, information about the device to be monitored would be found in the drive database.

=== START OF INFORMATION SECTION ===
Device is: In smartctl database [for details use: -P show]

The device should report a successful self-assessment test.

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

Please note: the drive database does not extend to NVMe devices. SMART support for NVMe devices is curently limited to a subset of features.

Step 5

Verify the SMART capabilities of the device.

$ sudo smartctl -c /dev/sda

The following output confirms that the device /dev/sda has both short and extended self-test capabilites.

=== START OF READ SMART DATA SECTION ===
SMART capabilities:            (0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:        (0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine 
recommended polling time: 	 (   2) minutes.
Extended self-test routine
recommended polling time: 	 (  85) minutes.

The output provides estimates for the duration of short and extended (long) self-test routines.

If the device is capable of self-tests

Use the following command to run a short self-test.

$ sudo smartctl -t short /dev/sda

Use the following command to run a long self-test.

$ sudo smartctl -t long /dev/sda

Display a list with the results of recent self-tests in reverse chronological order.

$ sudo smartctl -l selftest /dev/sda

All tests should have completed without error.

Step 6

Edit the default configuration /etc/smartd.conf and comment out any DEVICESCAN options, thus preventing smartd from attempting to search for attached devices indiscriminately.

On Debian 12, you can use the following command to comment out the DEVICESCAN option in the default configuration file.

$ sudo sed -i 's/DEVICESCAN -d removable -n standby -m root -M exec/#DEVICESCAN -d removable -n standby -m root -M exec/' /etc/smartd.conf

Step 7

Example configuration for smartd and SATA devices

For the device /dev/sda, the following configuration for monitoring the device with smartd would have to be added to /etc/smartd.conf.

/dev/sda -H -l error -l selftest -S on -s (L/../.././06|S/../.././18) -m root -M test

-H display the health status as reported by the device

-l error show the increase in the number of SMART errors since last check

-l selftest show the increase in the number of failed tests in the SMART Self-Test Log

-S on enable Attribute Autosave on startup

-s (L/../.././06|S/../.././18) schedule a long self-test between 06:00 and 07:00 daily and a short self-test between 18:00 and 19:00 daily

-m root local user root receives warning by email

-M test send a test email on startup

Example configuration for smartd and NVMe devices

Current versions of smartmontools offer experimental support for NVMe devices. In practice this means that only a limited, but still useful, feature set is available.

For the device /dev/nvme0, the following configuration for monitoring the device with smartd would have to be added to the end of /etc/smartd.conf.

/dev/nvme0 -H -l error -m root -M test

-H display the health status as reported by the device

-l error show the increase in the number of SMART errors since last check

-m root local user root receives warning by email

-M test send a test email on startup

Click to copy