How to: Shrink/Reclaim free virtual disk space from Virtual Machines on Proxmox VE (PVE) (Windows/Linux/Debian/Ubuntu/Kali Linux/RHEL/CentOS/Fedora etc.)

Pre-requirements

Following method only works for virtual machines (VM) that are satisfying these pre-requirements:

  • Thin-provisioned backing storage (qcow2 disk, thin-lvm, zfs, …)
  • Virtio-SCSI controller configured on guest.
  • Guest scsi disks with the discard option enabled [1]

Note: While changing provisioning types and Virtio-SCSI driver are not easy with existing virtual machines, but changing VM scsi disk’s discard options is simple, that means, if we appear to have an existing VM that is using thin-provisioned backing storage and Virtio-SCSI but “discard” options is not enabled/checked, we can simply find that VM and check that option, then we are good to follow the reset of this guide.

The Issue

When we are using qcow2 sparse virtual disks, we can reclaim free disk spaces which are not using by the virtual machine. How to trigger the VM/guest operating system to reclaim it for us though?

The Fix

1 Login to Proxmox VE web gui

2 Find the VM we want to reclaim the unused disk space for and click on it

3 Click on Hardware

4 Double click on the virtual hard’s virtual hard drive we want to reclaim unused space for

5 Make sure the “Discard” is checked

Proxmox VE - Discard option
Proxmox VE – Discard option

6 Start the VM

Once the VM is fully booted

6a For Linux/Debian/Ubuntu/Kali Linux/CentOS/RHEL/Fedora etc.

6a.1 We use following command to reclaim the unused disk space from a terminal

sudo fstrim -av

Once it’s done, we should be able to see the reclaimed disk space from Proxmox VE host (Only if there is unused space, if there is no unused space, we will not see any changes from Proxmox VE host’s disk space)

6a.2 We can also enable the automatic fstrim from the VM, so we do not need to do it manually everytime. Use following command to enable this feature

sudo systemctl enable fstrim.timer

6b For Windows

Usually the trim is enabled by default on Windows (Windows 7/2008R2 and up), we should not need to modify anything.

We can check if TRIM is enabled or not by using following command

fsutil behavior query DisableDeleteNotify

The output should be 0, otherwise, we can set it manually

fsutil behavior set DisableDeleteNotify 0

We can also trigger it manually, here is how.

First, we need to shutdown the Windows VM.

Then from the Proxmox VE web gui, find the Windows VM, Navigate to “Hardware”, double click on the virtual hard drive that we want to reclaim unused space from, make sure the “Discard” and “SSD emulation” are both checked, now start the Windows VM

Proxmox VE - Discard and SSD emulation checked
Proxmox VE – Discard and SSD emulation checked

When the Windows booted, we type “defrag” in start menu to search for “Defragment and Optimize Drives” program.

Windows 10 - Defragment and Optimize Drives
Windows 10 – Defragment and Optimize Drives

Click on it to launch it, then select the drive which we want to claim unused space from, click on “Optimize” button.

We now have manually reclaimed unused space from Windows VM

References

[1] “Shrink Qcow2 Disk Files – Proxmox VE”, Pve.proxmox.com, 2019. [Online]. Available: https://pve.proxmox.com/wiki/Shrink_Qcow2_Disk_Files


How to: Find/Show/List hidden directories/folders from Linux/Unix/Debian/Ubuntu/Kali Linux/RHEL/CentOS etc.

1 For System with Graphical User Interface (GUI) (Usually means we boot into an interface where we can see and use cursor)

1.1 Open the “Files” program (Like File Explorer in Windows)

1.2 For most of Linux distros use Ctrl + H key combination to show all hidden files and folders/directories (Use Ctrl + H again to hide them)

CentOS - File, showing hidden files and folders, directories
CentOS – File, showing hidden files and folders, directories

(In Linux folders and Files begin with the dot “.” hence the dotfile/dot file are hidden)

2 For Terminal

List all files and folders/directories (Including non-hidden and hidden)

ls -ahlp
Terminal - Showing all files and folders, directories (including non-hidden and hidden)
Terminal – Showing all files and folders, directories (including non-hidden and hidden)

-a: do not ignore entries starting with the dot “.”
-h: with -l and -s, print sizes like 1K 234M 2G etc.
-l: use a long listing format
-p: append / indicator to directories

List only hidden files

ls -ap | grep -v / | grep "^."
ls -ap | grep -v / | egrep "^."

-v: select non-matching lines
-v /: Inverse match everything with a slash “/”, so that only files are shwon

^. : Anything start with the dot “.”

Terminal - Show all hidden files
Terminal – Show all hidden files

List only hidden folders/directories

ls -ap | grep "^\..*/$"
ls -ap | egrep "^\..*/$"
Terminal - Showing all hidden folders, directories
Terminal – Showing all hidden folders, directories

List only hidden folders/directories without showing “./” and “../”

ls -Ap | grep "^\..*/$"
ls -Ap | egrep "^\..*/$"
Terminal - Showing only hidden folders, directories without anything else
Terminal – Showing only hidden folders, directories without anything else

Bonus

grep (Global Regular Expressions Print) and egrep (Extended Global Regular Expressions Print) have same function but different ways to interpret regular expression patterns.


How to: Find real path of symbolic links in Debian/Ubuntu etc.

e.g. The symbolic link is “/tmp/a.sh” which pointing to the real file “/tmp/test.sh”

1 Launch terminal or connect via SSH

2 Use following command we can see all symbolic links within a directory

ls -ahl /tmp | grep "\->"

Output

lrwxrwxrwx 1 [user] [group] [date] [time] a.sh -> test.sh

3 Alternatively we can use following command to check against single link file

readlink /tmp/a.sh

Output

test.sh

Want to know how to create symbolic/soft links and hard links?

Follow this guide: Quick Linux File Manipulation Commands Reference (and How to: Create Symbolic link/Soft link/Hard link)


How to: Reset forgotten CentOS 8 root password easily & quickly (CentOs 8 on UEFI) (& Red Hat Enterprise Linux/RHEL)

CentOS 8

1 Start the system

2 At the grub menu screen, press Up/Down Arrow keys on the keyboard to stop count down timer

CentOS 8 grub menu
CentOS 8 grub menu

3 Highlight the default boot item, usually the first one, press “e” key on the keyboard, to edit the boot option

The default content is as shown

Default CentOS 8 UEFI grub menu
Default CentOS 8 UEFI grub menu
load_video
set gfx_payload=keep
insmod gzio
linux ($root)/vmlinuz-4.18.0-147.5.1.el8_1.x86_64 root=/dev/mapper/cl-root ro \
crashkernel=auto resume=/dev/mapper/cl-swap rd.lvm.lv=cl/root rd.lvm.lv=cl/swa\
p rhgb quiet
initrd ($root)/initramfs-4.18.0-147.5.1.el8_1.x86_64.img $tuned_initrd

We add “rd.break” at the end of “quiet” before “initrd”

Before

Before editing
Before editing

After

After editing
After editing
load_video
set gfx_payload=keep
insmod gzio
linux ($root)/vmlinuz-4.18.0-147.5.1.el8_1.x86_64 root=/dev/mapper/cl-root ro \
crashkernel=auto resume=/dev/mapper/cl-swap rd.lvm.lv=cl/root rd.lvm.lv=cl/swa\
p rhgb quiet rd.break
initrd ($root)/initramfs-4.18.0-147.5.1.el8_1.x86_64.img $tuned_initrd

4 Use Ctrl + X key combination to start/boot the system

Booted
Booted

5 Input following commands

mount -o remount,rw /sysroot
chroot /sysroot
passwd

We will be asked for setting new root password, to be safe, we should input a password that is strong enough, at least 8 characters, can’t be among the common password list.

Otherwise it will complain “BAD PASSWORD: The password is shorter than 8 characters” or “BAD PASSWORD: The password fails the dictionary check – it is too simplistic/systematic”, we can ignore it and continue with the weak password if it’s just for testing purpose.

BAD PASSWORD: The password is shorter than 8 characters
BAD PASSWORD: The password is shorter than 8 characters
BAD PASSWORD: The password fails the dictionary check - it is too simplistic/systematic
BAD PASSWORD: The password fails the dictionary check – it is too simplistic/systematic

Continue with following command

touch /.autorelabel
exit
reboot

FYI (Complete commands)

mount -o remount,rw /sysroot
chroot /sysroot
passwd
touch /.autorelabel
exit
reboot

6 We now have successfully rest forgotten password for CentOS 8 on UEFI, now we can login with the new password.

Red Hat Enterprise Linux 7 and 8/RHEL 7 and 8

The procedure is very similar with CentOS 8.

The only difference is, instead of adding the “rd.break” at the end of “quiet” we need to find “quiet LANG=en_US.\ UTF-8″ then at the end add ” rd.break”, again, the process is very similar.

The rest will be exactly the same