Integrating Barman with Netbackup

The goal of this article is to highlight how to integrate Barman with Veritas NetBackup in order to meet business continuity requirements.

Veritas NetBackup is the enterprise class backup and recovery solution used in many organisations. While it is not officially supported by 2ndQuadrant, this article proposes a view ideas on how to integrate backup and recovery of your PostgreSQL databases in enterprise wide backup processes handled by Netbackup.

Barman is an open source tool developed by 2ndQuadrant that is specialised in hot backups of PostgreSQL databases with high goals of business continuity, including zero data loss - that is RPO (Recovery Point Objective) = 0. Barman also assists administrators in the recovery phase.

The goal of this document is to highlight how to integrate the two technologies in order to meet business continuity requirements.

The general idea is to use Barman as primary backup and recovery tool for PostgreSQL databases in the short period, and rely on NetBackup as both backup of Barman and source of restore in the mid and long term, in accordance with the organisations' retention policies.

Under common circumstances, one Barman installation is enough to control backup and recovery of multiple PostgreSQL instances in the same data centre.

Barman is one of the most used backup tools in the PostgreSQL ecosystem, and it is fully supported by 2ndQuadrant.

Documentation of Barman can be found at https://docs.pgbarman.org/.

Support customers can also browse our Knowledge Base for resources about Barman at https://access.2ndquadrant.com/kb/?q=product:barman.

Our proposed backup strategy is to have two layers of backups, based on retention policies:

• First layer, short term: Barman with short retention policies (e.g.: 2 weeks)
• Second layer, mid-long term: NetBackup backing up the barman_home folder, inside the corporate retention policies

The idea is to use Barman to recover after one of the most likely events to happen:

• full recovery (up to the last available transaction)
• point in time recovery within Barman's point of recoverability - according to our experience, most of these events are limited to a few days or a week range

Barman's support for streaming replication, as well as integration with standby servers is indeed the state of the art for high availability and disaster recovery objectives of PostgreSQL databases. Barman has also native monitoring capabilities, for example with Icinga/Nagios, that help increase robustness and reliability of your PostgreSQL enterprise environment.

NetBackup will be configured to backup files contained in the barman_home folder (Pathname), in an incremental way.

The two main points regarding backup policies of PostgreSQL databases in Barman are:

• Frequency of full base backups
• Retention policies definition

The first item is taken care by a cronjob installed on the Barman server, as detailed in our Knowledge Base article at https://access.2ndquadrant.com/kb/a/hot-physical-backups-with-barman/. Weekly full base backups are recommended for most common scenarios.

Retention policies can be defined in the main configuration file of Barman using two methods:

• Redundancy: number of base backups
• Recovery window: based on point of recoverability, such as "RECOVERY WINDOW OF 2 WEEKS"

The proposed backup strategy suggests 4 possible use cases for recovery:

1. Full recovery of a PostgreSQL instance
2. Point In Time Recovery (PITR) within Barman's point of recoverability
3. Point In Time Recovery outside Barman's point of recoverability (within NetBackup retention policies)
4. Full recovery of the Barman server

When a PostgreSQL restore is required for any reason, Barman is the most effective way to restore an instance both in terms of RTO and RPO (especially in case of streaming replication). The barman recover command can be used to restore the target instance starting from the last base backup available and until the latest transaction that's been correctly archived in Barman.

Recovery relies on a single command to be executed on the Barman server:

barman recover <server_id> <backup_id> --ssh-remote-command 'COMMAND' <target_dir>

Where:

• <server_id> is the ID of the server (the same defined within brackets in the per server configuration file).
• <backup_id> is the ID of the previous complete base backup (as timestamp).
• <target_dir> is the path to the target directory in which PostgreSQL will be restored.
• --ssh-remote-command is the option to specify how barman connects to the remote hosts (is an SSH connection to the postgres system user: ssh postgres@<postgresql_hostname> - RSA Keys exchange is required).

In case of Point In Time Recovery in a range comprised between the first available backup in Barman and the last available transaction, it is advised once again to rely on Barman for recovery.

The command to be executed is identical to case 1. It just needs to specify the recovery target. Although there are several possibilities, recovery target is specified in the form of a timestamp, using the --target-time option. Recovery target instructs PostgreSQL to recover up to a specific point in time.

For example:

barman recover <server_id> <backup_id> --ssh-remote-command 'COMMAND' <target_dir> --target-time '2018-07-23 07:25:12.362'

Where:

• --target-time is the option to specify the moment up to which PostgreSQL will recover the instance.

In this scenario NetBackup tool comes in handy when the required backup has already been removed automatically by Barman cron activity to respect retention policies.

NetBackup in this case can provide an older snapshot of the Barman home in which the required backup can be found.

IMPORTANT: Barman currently does not allow users to specify directories containing older backup catalogs to be used in this occasions (such as 'recovery_snapshot_dirs'). If needed, we can work on such a feature.

The first step is to recover the barman_home on a separate folder in the Barman server. Alternatively, the snapshot can be mounted on the Barman server.

The second step is to temporarily disable retention policies for the server that needs to be recovered, by setting them to an empty string.

Once the required NetBackup snapshot is available, the target backup and the associated WAL files need to be copied back inside the correct barman directory, since Barman also removes WAL files that are associated with the oldest backup.

As for example, the target directories for the "main" PostgreSQL instance backup should be:

• /path/to/barman/home/main/base is the directory in which <backup_id> (directory with timestamp as name) should be copied.
• /path/to/barman/home/main/wals/ is the directory in which every needed WAL files should be copied from the snapshot.

Once the copy is done, PITR procedure is the same as outlined in the previous sections.

After the recovery is completed, unmount the snapshot volume (where applicable) and restore retention policies (this will automatically trigger removal of old files).

Although very unlikely, it sometimes can happen that the server containing Barman is damaged or its content lost, and a full Barman server restore is needed.

The procedure is quite straightforward:

1. Re-deploy the Linux server with the same configuration as the previous one
2. Copy the latest backup of the barman_home directory from NetBackup in the original location
3. Re-execute a full backup of all instances (barman backup all will suffice)

The last step is required because the continuous archive of WAL files is inevitably interrupted by the incident that affected the Barman server. Indeed, there is no guarantee (it is actually very unlikely) that the latest NetBackup snapshot contains the latest WAL file that was received by Barman before the incident. For this reason, 2ndQuadrant has developed hook scripts for Barman that allow users to reliably relay WAL files on additional data sources and locations, including for example encrypted AWS S3. These hook scripts are part of the Barman Utils package, available for our support customers via our RPM/APT repositories.

Therefore it is advised to perform a new full base backup of all servers hosted in the Barman server, once the restore of the barman_home from NetBackup has finished, through the barman backup all command.