DR with remote cluster

[ciroiriarte]

I’ve read about the option of doing backups to a NAS and also replicating (?) to a second/remote cluster.

Is this copy to a remote cluster a second backup job or is it leveraging LizardFS replication features?. I would like to understand if it’s a synchronous/semi-synchronous replication or if it’s more like a backup job (with hour-level RPO)

Also, what’s the expected procedure to start VMs in the destination cluster. For example, is there any contextualization trick to change the IPs?, is the same subnet expected as operational in the second site?

[penzoiders]

It’s not leveraging DFS replicas for remote backups, It’s based on Borg and it comes with dedup/compression/encryption.

You can schedule jobs to sync a snapshot backup to the remote site. any time you want, according to the feasibility (you should see how long it takes depending on your bandwidth and data weight).

To spin up a vm on the remote one you should prepare the templates and virtual networks, then you can recover the images from any consistent point in time provided by the Borg based backup.

At the moment we’re working on a 1-click cluster restore that will take the last status of the borg-based backup (or another point in time of the same set) and restores the complete production cluster’s last saved state to the backup one, all configs, templates, networks will be as in the source cluster.
This feature is not yet available and is realistic to see it in 2019.

you’ll just need to spin up VMs then.

[penzoiders]

best thing is anyway to shape custom templates for DR, as well as custom DR virtual networks and use contextualization to speed up restore procedures and avoid human error.

This is what commonly in place where DR clusters are used.

Prod --borg–> Backup
DR Templates/Nets on Backup

in case of need —> restore selected images and spin 'em up with desired DR Template/Net

[ciroiriarte]

So, is it correct to think that destination site will need more storage to receive the backup jobs and restore the machines (expecting same capacity por production as the main site)?.

I’m thinking that in that case, it might make more sense to just do the “backup to NAS” and locate the NAS on the second site (having cheap disks in the NAS, and SSD for production)

[penzoiders]

usually at the destination with compression and dedup you’ll end up using less space than provisioned (even if you keep many days of history).

let me throw here a real life example of a 30 days worth NodeWeaver backup repo:

---

Original size
This archive: 3.68 TB --> NodeWeaver Images Allocated Space at source cluster
All archives: 121.62 TB --> what would it be to extract all backups from the repository

Compressed size
This archive: 857.45 GB --> All NodeWeaver Images on the source cluster once compressed
All archives: 24.58 TB --> all the backups would take this space compressed (from 121.62TB to 24.58TB)

Deduplicated size
This archive: 28.02 GB --> what has actually been sent from the source to backup
All archives: 1.73 TB --> total occupied space on backup storage (30 days, day by day recovery capable, of the whole 3.68 TB provisioned storage at source)

---

Of course you may need a bit more space if you want to extract all of your data, but in the end you don’t need to oversize your backup.

Note: in this example VM Images have of course free space, can’t tell you exactly how much, but there are about 30 VMs cluster doing active work, running on 3 all-flash nodes in production and 2 mainly-rotational+small-flashes(just in case) nodes for backup

[ciroiriarte]

Thanks!. Given this context “the idea to restore on a remote cluster”, is then any difference between having a cluster and a NAS as backup destination?

[penzoiders]

Restore on backup cluster from local DFS repo (sent from source cluster) should be faster than retrieving from NAS.

If NAS is well performing and you have 10gbit network may be same speed as in cluster Borg repo

[ciroiriarte]

Oh, forgot to ask. Is the backup destination allocated within the DFS?, or needs a regular filesystem dedicated to that?

[penzoiders]

If you backup to another NodeWeaver cluster the backup is held in the DFS of the destination cluster. If you use a NAS target the underlying filesystem can be whatever.