Besides many existing tools, i’ve discovered znapzend to be a very powerful tool to manage automatic snapshots including retention and cleanup for time bases snapshot plans. I’m using it with the Proxmox builtin ZFS on Linux to create snaphots and copy them to local and remote disks. This allows me to build some pseudo HA cluster without maintaining things like Ceph as i have the VM filesystems available on other machines.
ZnapZend is a ZFS centric backup tool. It relies on snapshot, send and receive todo its work. It has the built-in ability to to manage both local snapshots as well as remote copies by thining them out as time progresses.
The ZnapZend configuration is stored as properties in the ZFS filesystem itself.
Just install it using dpkg (requires mbuffer):
sudo apt-get install mbuffer sudo dpkg -i znapzend_0.14.1-1_amd64.deb
It is a good idea to create your datasets as atomic as possible. Do not use a single tank to hold all data. I do a dataset for VMs, a dataset for fileserver, etc. This makes it easy to migrate things later if the disks get full or the demands change. This also allows me to create a recursive snapshot task for the everchanging vm dataset, so i don’t need to add plans for new VMs.
Let’s assume having a zfs pool called tank. Create a dataset like tank/vm-store on which we can setup the plan without disturbing another dataset like tank/git-repositories or tank/vm-templates
The plan defines how the snapshots are being taken. It defines independant time slots for source and optional one or multiple targets. So we can easily define a plan like this:
On source pool take a snapshot:
- every hour and keep it 24 hours
- every day and keep it for 6 days
- every week and keep it 4 weeks
- every month and keep it 12 months
Second, we take care of a exact copy on another local backup disk.
Third, we can send all of this to another machine to have a backup outside our system. It could be the same plan (like i do) or different (e.g. get rid of the hourly and only store the daily snapshots).
Examples can be found on github.
The plan from above will look something like this using
*** backup plan: tank/vm-data *** dst_a = local-backup/tank/vm-data dst_a_plan = 6days=>1day,4weeks=>1week dst_b = root@backup-server:backup/proxmox/vm-data dst_b_plan = 6days=>1day,4weeks=>1week enabled = on mbuffer = /usr/bin/mbuffer mbuffer_size = 1G post_znap_cmd = off pre_znap_cmd = off recursive = on src = tank/vm-data src_plan = 24hours=>1hour,6days=>1day,4weeks=>1week tsformat = %Y-%m-%d-%H%M%S