Geo-redundancy of PostgreSQL database backups with Barman

Gabriele Bartolini
Gabriele Bartolini

This article covers the concepts and configuration examples for the Barman geo-redundancy feature, which allows Barman to synchronize data from another Barman server.

Background

Since Barman 2.6, Barman has support for geo-redundancy, meaning that Barman can copy from another Barman instance, not just a PostgreSQL database.

Geographic redundancy (or simply geo-redundancy) is a property of a system that replicates data from one site (primary) to a geographically distant location as redundancy, in case the primary system becomes inaccessible or is lost. From version 2.6, it is possible to configure Barman so that the primary source of backups for a server can also be another Barman instance, not just a PostgreSQL database server as before.

Briefly, you can define a server in your Barman instance (passive, according to the Barman lingo), and map it to a server defined in another Barman instance (primary).

All you need is an SSH connection between the Barman user in the primary server and the passive one. Barman will then use the rsync copy method to synchronize itself with the origin server, copying both backups and related WAL files, in an asynchronous way. Because Barman shares the same rsync method, geo-redundancy can benefit from key features such as parallel copy and network compression. Incremental backup will be included in future releases.

Geo-redundancy is based on just one configuration option: primary_ssh_command.

Our existing scenario

To explain how geo-redundancy works, we will use the following example scenario. We keep it very simple for now.

We have two identical data centers, one in Europe and one in the US, each with a PostgreSQL database server and a Barman server:

  • Europe
  • PostgreSQL server: eric
  • Barman server: jeff, backing up the database server hosted on eric
  • US
  • PostgreSQL server: duane
  • Barman server: gregg, backing up the database server hosted on duane

Let’s have a look at how jeff is configured to backup eric, by reading the content of the /etc/barman.d/eric.conf file:

[eric]
description = Main European PostgreSQL server 'Eric'
conninfo = user=barman-jeff dbname=postgres host=eric
ssh_command = ssh postgres@eric
backup_method = rsync
parallel_jobs = 4
retention_policy = RECOVERY WINDOW OF 2 WEEKS
last_backup_maximum_age = 8 DAYS
archiver = on
streaming_archiver = true
slot_name = barman_streaming_jeff

For the sake of simplicity, we skip the configuration of duane in gregg server as it is identical to eric's in jeff.

Let's assume that previously we have had this configuration in production for a few years (this goes beyond the scope of this article indeed).

Europe
eric (Postgres) -> jeff (Barman, eric's backup)

US
duane (Postgres) -> gregg (Barman, duane's backup)

The European Barman backed up the European PostgreSQL server, and the US Barman backed up the US PostgreSQL server.

Adding geo-redundancy

Our system is then upgraded to Barman 2.6 or above, and we can add geo-redundancy into the system.

We will tell our Barman servers to relay their backups on the other, with a higher retention policy.

As a first step, we need to exchange SSH keys between the barman users (you can find more information about this process on Barman documentation).

We also need to make sure that compression method is the same between the two systems (this is typically set as a global option in the /etc/barman.conf file).

Before geo-redundancy:

US | Europe
|
duane (Postgres) | eric (Postgres)
|
gregg (Barman) | jeff (Barman)
duane's backup | eric's backup
  • gregg Barman only stores duane's backup.
  • jeff Barman only stores eric's backup.

Geo-redundancy sharing:

US | Europe
|
gregg (Barman) --pull--> duane's backup
|
eric's backup <--pull-- jeff (Barman)
|

Resulting state:

US | Europe
|
duane (Postgres) | eric (Postgres)
|
gregg (Barman) | jeff (Barman)
duane's backup | eric's backup
eric's backup | duane's backup
  • gregg Barman now stores both duane's and eric's backups.
  • jeff Barman now stores both eric's and duane's backups.

Creating the Europe <-- US pull

Within Barman jeff, we define duane as a passive server. jeff will be storing backups of duane.

To do this, we create a file called /etc/barman.d/duane.conf with the following content:

[duane]
description = Relay of main US PostgreSQL server 'Duane'
primary_ssh_command = ssh gregg
retention_policy = RECOVERY WINDOW OF 1 MONTH

This config allows jeff to pull the duane backup from gregg, and store it.

As you may have noticed, we declare a longer retention policy in the redundant site (one month instead of two weeks).

If you type barman list-servers, you will get something similar to:

duane - Relay of main US PostgreSQL server 'Duane' (Passive)
eric - Main European PostgreSQL server 'Eric'

The cron command is responsible for synchronizing backups and WAL files. This happens by transparently invoking two commands: sync-backup and sync-wals, which both rely on another command called sync-info (used to poll information from the remote server). If you installed Barman using the RPM or DEB package, a cron entry running on every minute will be created for you.

It's also possible to directly execute the aforementioned commands to check the current status (barman sync-info), as well as manually execute commands barman sync-backup and barman sync-wals to copy both the latest backups and latest WAL files from the primary Barman server. For more details on manual synchronization, please check the documentation.

The Barman installation on jeff will now check its catalogue for the duane server backups, against the ones located on the Barman instance gregg. If there is a difference, it will pull the differences. First it copies the backup files (from the most recent one), and then the related WAL files.

One peek at the logs should unveil that Barman has started to synchronize its content with the origin one. To verify that backups are being relayed, type:

barman list-backup duane

When a passive Barman server is copying from the primary Barman, you will see an output like this:

duane 20190217T113619 - SYNCING

When the synchronization is completed, you will see the familiar output of the list-backup command:

duane 20190217T113619 - Sun Feb 17 11:36:23 2019 - Size: 543.6 MiB - WAL Size: 450.3 MiB

Creating the US <-- Europe pull

We will now do the reverse.

Within Barman gregg, we define eric as a passive server. gregg will be storing backups of eric.

To achieve this, we create a file called /etc/barman.d/eric.conf with the following content:

[eric]
description = Relay of main European PostgreSQL server 'Eric'
primary_ssh_command = ssh jeff
retention_policy = RECOVERY WINDOW OF 1 MONTH

This config allows gregg to pull the eric's' backup from jeff, and store it.

What's more

The above scenario can be enhanced by adding a standby server in the same data center, and/or in the remote one, as well as by making use of the get-wal feature through barman-wal-restore.

The geo-redundancy feature increases the flexibility of Barman for disaster recovery of PostgreSQL databases, with better recovery point objectives and business continuity effectiveness at lower costs.

Should one of your Barman servers go down, you have another copy of your backup data that you can use for disaster recovery – of course with a slightly higher RPO (recovery point objective).

Was this article helpful?

0 out of 0 found this helpful