Archive for Sierpień 2014
As we all know – disk space isn’t free. If you allocate disk space to your virtual machine, you always want to allocate proper, well-balanced size to shorten future downtimes. But what is proper and balanced you ask? There is no short answer to this, but if you’re IT person, you always will try to allocate more than is needed right at the moment. The decision is : should I use static or dynamic disks?
For me, there is no really need to use static disks anymore as there is no real speed difference. So, the real pro using dynamic disk is their smaller size.
In production environment, using Microsoft’s hypervisor, we can expect our VM to grow over time. During the nature of virtual machines and virtual disks, the real data machine uses are not the same numbers as virtual disk sizes. It is happening due to hypervisor behaviour – deleting all data from OS in virtual machine is no more than throwing away and index card from your book. The written data are still there, so there is no easy, automated way to compact a virtual disk.
Hypervisor however, tries to allocate in a first place, blocks marked as used. So the real expanding is happening when you’re constantly changing your data, without deleting them first. The great example is with databases, log and temporary trash disks and just plain and simple oversizing. I’ve had a case when relativly small machine (20 GB CentOS with a 600 GB VD) just grew over few days filling whole 600 GB of data because of logs set to DEBUG.
So what will be the cons and of virtual disk growing overtime?
- obviously, more space allocated at expensive storage for VMs
- more obviously, when you’re using any kind of backup software, you’re forced to write and store in multiple copies the data you really doesn’t need
- CBT (Change Block Tracking) Table is bigger to process, with every move
- more network traffic with every backup job you have
- live ‚shared nothing’ migration times grows just out of proportions. If you have a small machine with 20GB data on 600 GB sized disk, you will have to transfer this whole fatso over you network to other machine. Even with compression set at live migration, it is just really unwanted.
So what can we do? We can just zero the data you’re not using and Hyper-v cmdlets will take the rest.
You have to plan downtime for the machine, but according to my tests, zeroing machines with 600 GB took 15 to 30 minutes. With smaller sizes it is just matter of single minutes.
Before you go:
– plan your downtime, compacting should not to be interrupted
– take extra caution. Zeroing important data or uneeded data it’s just a matter of mistake.
– delete all snapshots/checkpoints
– make sure you converted VHD to VHDX (optional)
– make sure your disk is set as dynamic disk
– make sure you have enough room for compacting or resizing. Remember – if you have 600 GB virtual disk, during this process it may grow to this size.
– remember that compacting deletes CBT table for backup software – next backup will be a Full Backup.
The most usable zeroing ways I found was:
– longer downtime
– more accurate (no more background write)
– faster (no more background write)
– smaller risk of processes failure due to ‚out of space’ event
- zerofree for Linux ext2/ext3/ext4 disks
- ntfswipe for Windows based disks
- dd for both Linux and Windows based disks or exotic fs (btrfs/xfs)
– less accurate (background write is still happening)
– machines became slower or unresponsible
– slower process
– risk of applications failing due to ‚out of space’ events
– smaller downtime
– SDelete or CCleaner for Windows based disks
– dd for Linux based disks
Phase I – Zeroing Offline
Offline zeroing (done with System Rescue CD 4.x)
- Delete all uneeded data (logs, temps, bigfiles, dowloads, caches)
- Umount all ext2/3/4/ntfs volumes
- for NTFS volume: ntfswipe -av /dev/volume_name
- for ext2/3/4 volume: zerofree -v /dev/volume-name
- for LVM: vgchange -a y;zerofree -v /dev/mapper/volume_name
Zeroing swap space:
# swapoff -a # free total used free shared buffers cached Mem: 8056340 2643132 5413208 155072 606916 914796 -/+ buffers/cache: 1121420 6934920 Swap: 0 0 0 # blkid |grep swap /dev/sdb2: UUID="adad0488-3b67-4444-b792-6a1a775b8821" TYPE="swap"
# dd if=/dev/zero|pv -treb|dd of=/dev/sdb2 bs=8192 dd: error writing ‘/dev/sdb2’: No space left on device 1,91GB 0:01:04 [30,3MB/s] 249981+4 records in 249980+4 records out 2047868928 bytes (2,0 GB) copied, 67,931 s, 30,1 MB/s
#mkswap /dev/sdb2 -U "adad0488-3b67-4444-b792-6a1a775b8821" Setting up swapspace version 1, size = 1999868 KiB no label, UUID=adad0488-3b67-4444-b792-6a1a775b8821
# swapon -a # free total used free shared buffers cached Mem: 8056340 3309648 4746692 159844 1112232 919708 -/+ buffers/cache: 1277708 6778632 Swap: 1999868 0 1999868
Phase I – Online zeroing
1. Delete all uneeded data (logs, temps, bigfiles, dowloads, caches)
for Windows machine
sdelete -z letter:
#dd if=/dev/zero|pv -treb|dd of=/file.zero bs=4096;sync;sync;rm -rfv /file.zero;sync
Phase II – Compacting
1. Shutdown the machine
For System Center VMM 2012 R1/R2
Optimize-VHD -Path path_to_vhdx_file.vhdx -Mode Full
System Rescue CD – http://www.sysresccd.org/Download
SDelete – http://technet.microsoft.com/en-us/sysinternals/bb897443.aspx
CCleaner Portable – https://www.piriform.com/ccleaner/builds
zerofree – http://manned.org/zerofree/00be91ab