[HCoop-Discuss] how our backups work[ed]

[Adam Megacz]

"docelic" <docelic at mire.hcoop.net> writes:
> 2) megacz or mwolson, what's our situation re. homedir backup? I've
>   browsed through /afs/hcoop.net/common/etc/scripts/*backup* and it seems
>   to me backups are done holding 'hcoop' tokens instead of
>   'system:backup'?

What I write below describes the state things were in when I left them
a few months ago.  I do not know what may have changed since then.

All volumes are backed up using the "dump" mechanism.  This is sort of
the AFS equivalent of "dd if=/dev/hda1 of=mybackup" -- it is an action
performed as the root user, directly on the fileserver machine, and
reads bits essentially right off the disk (ie not via the FileServer
APIs), so it bypasses all security checks -- just like doing a "dd" on
a raw partition bypasses the permission bits on unix.  ACLs have no
effect on what is or is not backed up.

Dump files (again, basically a gigantic "disk image" of the AFS
volume) are compressed, encrypted, and then stored offsite (currently
on /afs/megacz.com/, which lives on gentzen.megacz.com).

A major problem right now is the sheer size of the backups; a full
backup is ~24GB (compressed), so the backups are failing on a regular
basis right now because they're hitting the disk limit cap on the
volume where they are stored.  We seem to be getting a complete,
successful backup about once every other week.

I've been trying out xdelta3 as a way to do incremental backups, and
it's been working quite well for almost four months now.

  /afs/megacz.com/srv/bin/dump.sh

The only catch is that I trust the machine on which I store the
backups.  Doing backups to an untrusted host requires the "full and
forward" style, which has disadvantages compared to "latest and
reverse".

  - a