February 7, 2016

ZFS snapshotting using znapzend

Introduction

Besides many existing tools, i’ve discovered znapzend to be a very powerful tool to manage automatic snapshots including retention and cleanup for time bases snapshot plans. I’m using it with the Proxmox builtin ZFS on Linux to create snaphots and copy them to local and remote disks. This allows me to build some pseudo HA cluster without maintaining things like Ceph as i have the VM filesystems available on other machines.

ZnapZend is a ZFS centric backup tool. It relies on snapshot, send and receive todo its work. It has the built-in ability to to manage both local snapshots as well as remote copies by thining them out as time progresses.

The ZnapZend configuration is stored as properties in the ZFS filesystem itself.

Installation

You might just check out the Github page (packaging). To make things easier to deploy in our Proxmox cluster, i’ve created a deb file for 64bit Jessie in my repo.

Just install it using dpkg (requires mbuffer):

sudo apt-get install mbuffer
sudo dpkg -i znapzend_0.14.1-1_amd64.deb

Setup hints

It is a good idea to create your datasets as atomic as possible. Do not use a single tank to hold all data. I do a dataset for VMs, a dataset for fileserver, etc. This makes it easy to migrate things later if the disks get full or the demands change. This also allows me to create a recursive snapshot task for the everchanging vm dataset, so i don’t need to add plans for new VMs.

Let’s assume having a zfs pool called tank. Create a dataset like tank/vm-store on which we can setup the plan without disturbing another dataset like tank/git-repositories or tank/vm-templates

Plans

The plan defines how the snapshots are being taken. It defines independant time slots for source and optional one or multiple targets. So we can easily define a plan like this:

On source pool take a snapshot:

  • every hour and keep it 24 hours
  • every day and keep it for 6 days
  • every week and keep it 4 weeks
  • every month and keep it 12 months

Second, we take care of a exact copy on another local backup disk.

Third, we can send all of this to another machine to have a backup outside our system. It could be the same plan (like i do) or different (e.g. get rid of the hourly and only store the daily snapshots).

Examples can be found on github.

The plan from above will look something like this using znapzendzetup list

*** backup plan: tank/vm-data ***
dst_a           = local-backup/tank/vm-data
dst_a_plan      = 6days=>1day,4weeks=>1week
dst_b           = root@backup-server:backup/proxmox/vm-data
dst_b_plan      = 6days=>1day,4weeks=>1week
enabled         = on
mbuffer         = /usr/bin/mbuffer
mbuffer_size    = 1G
post_znap_cmd   = off
pre_znap_cmd    = off
recursive       = on
src             = tank/vm-data
src_plan        = 24hours=>1hour,6days=>1day,4weeks=>1week
tsformat        = %Y-%m-%d-%H%M%S