What is an in­cre­ment­al backup?

In­cre­ment­al backups allow for con­tinu­ous storage of changing data sets. This minimizes the time, bandwidth, and storage re­quire­ments compared to repeated full backups. In­cre­ment­al backups begin with a full backup, followed by a chain of smaller in­cre­ment­al backups, but results in higher com­plex­ity of backup and restore processes. Special software is used to cope with this.

An in­cre­ment­al backup is a backup method that is often used as part of a wider backup strategy. Unlike a full backup, it does not create a complete copy of the dataset to be backed up. Instead, only changes made to files as of the last backup are saved.

Unlike a dif­fer­en­tial backup, the re­con­cili­ation of changes does not ne­ces­sar­ily refer to the last full backup. Rather, the changes since the last full or in­cre­ment­al backup are saved.

In general, there are two broad cat­egor­ies of ap­proaches to in­cre­ment­al data pro­tec­tion:

1. The gran­u­lar­ity of changes
2. The use of the strategy to be employed

Usually, hybrid ap­proaches are used. For example, the popular tool Rsync creates `block-level synthetic full backups´ in normal use. Let’s look at the different types of in­cre­ment­al backup in detail.

Digitally stored data consists of almost in­fin­itely long chains of zeros and ones. These bits are grouped into logical units, of which files are the best known. Below file level, however, there are `blocks´ and `bytes´ as logical groupings of in­di­vidu­al bits.

Comparing changes between the last backup and the current state of the dataset can refer to different levels. The closer the matching of changes gets to the in­di­vidu­al bits, the more efficient the in­cre­ment­al backup becomes. While files vary in size, blocks and bytes have a defined, fixed size. For example, a byte comprises eight con­sec­ut­ive bits; blocks are usually between 512 and 4,096 bytes long.

A file-level in­cre­ment­al backup only dis­tin­guishes whether a file has been changed. The extent of the change is not taken into account. If even a single bit has been changed, the entire file is backed up again in the course of the in­cre­ment­al backup. To il­lus­trate this, imagine a large book ma­nu­script that is stored as a single file on a data storage device. If even a single letter is changed, the entire ma­nu­script is backed up again as part of a file-level in­cre­ment­al backup.

A file-level in­cre­ment­al backup is the easiest to implement because file systems log the modi­fic­a­tion date of files. To reconcile the changes, it is suf­fi­cient to compare the timestamps of the last modi­fic­a­tion of a file on the source and target systems. If the timestamp on the source system is newer, the file has been modified and must be backed up again. Thus, file-level in­cre­ment­al backups are in­ef­fi­cient for small changes to large files.

The term `block´ ori­gin­ates from data storage. Hard disks and other mass storage devices organise sections of con­sec­ut­ive bytes as a con­tigu­ous logical area. When creating the block-level in­cre­ment­al backup, only the changed blocks are backed up. The advantage is the smaller size of the data to be trans­ferred and stored. Thus, only the changed sections of in­di­vidu­al files are backed up.

The main dis­ad­vant­age is that a mechanism is needed to log which blocks have changed. Some storage tech­no­lo­gies are able to manage this in­form­a­tion and thus can implement block-level in­cre­ment­al backups. Otherwise, the full backup at block level would need to be compared with the current state to detect changes.

A byte-level in­cre­ment­al backup is an even more detailed version than a block-level in­cre­ment­al backup. Only the changed bytes of a block are backed up. In terms of the time required and the storage space needed, the byte-level in­cre­ment­al backup may be sig­ni­fic­antly more efficient. However, the process is more demanding. This is because a journal of the byte-level changes is required to identify the changed bytes. The ad­di­tion­al effort for byte-level in­cre­ment­al backups is useful for selective changes to very large files.

In­cre­ment­al backups are part of a chain of in­di­vidu­al backups that contain changes to the previous state. At the beginning of the chain there is always a full backup. There are various methods of con­struct­ing the chain and creating new full backups. This has ad­vant­ages in terms of com­plex­ity and duration of backup and restore processes. Here are some strategies for creating in­cre­ment­al backups.

A tra­di­tion­al in­cre­ment­al backup strategy requires periodic full backups. Creating a full backup is time-consuming and slow. With synthetic full backups, a new full backup is created. The original full backup and the sub­sequent in­cre­ment­al backups are used. This saves the copying of the entire dataset from the source system that would otherwise be required when creating a full backup. The process is also known as `forward in­cre­ment­al backup´.

The in­cre­ment­al forever backup is optimised for backing up hard disks and similar media. Initially, a full backup is created; af­ter­wards only the changed blocks are stored in a `forever´ con­tinu­ous chain. Unlike a synthetic full or a reverse in­cre­ment­al backup, no further full backups are created after the initial one. This saves storage space and minimises the data trans­ferred during the copy op­er­a­tions.

The reverse in­cre­ment­al backup works similar to the synthetic full backup. After each in­cre­ment­al backup, changes are merged with the last full backup. This means a full backup exists by the end of the backup chain. It is true that creating a reverse in­cre­ment­al backup takes longer than a forward in­cre­ment­al backup. However, if necessary, the source system can be restored without delay, since the last current status is already available as a complete copy.

An enhanced in­cre­ment­al backup can be of any type. A special feature is that it also detects changes to files caused by moving or renaming them. Provided large files are involved, this boosts ef­fi­ciency.

The biggest advantage of in­cre­ment­al backups is that in­di­vidu­al backups are small. Since only the dif­fer­ences as of the last backup are saved, the backup process requires re­l­at­ively little time, bandwidth, and storage space. The pre­requis­ite is that the period between the in­di­vidu­al backups is kept short. Otherwise, large amounts of changed data ac­cu­mu­late and have to be trans­ferred.

The inherent advantage of in­cre­ment­al backups is well il­lus­trated by the term `backup window´. This is the time period within which a backup can be created without dis­rupt­ing op­er­a­tions. If the amount of data is so large that the copying process would take longer than the backup window allows, it becomes im­possible to create a full backup during operation. In that case it is advisable to create small in­cre­ment­al backups at short intervals.

The immediate dis­ad­vant­age of in­cre­ment­al backups is that spe­cial­ised software is usually necessary. Planning the backup strategy also requires more effort than creating full backups. Restoring the original state from the backups is similar. Because data is spread across several backups, the process is more complex. There is also a higher risk of data loss. If one in­cre­ment­al backup in a chain is damaged, the integrity of all sub­sequent backups is affected.

In­cre­ment­al backups are primarily aimed at min­im­ising the storage re­quire­ments and time needed to create a backup. They are therefore always used when the repeated creation of complete backups would be lo­gist­ic­ally non­sensic­al. Let’s look at a few scenarios in detail.

To create a backup on Mac, it is best to use the macOS built-in backup tool “Time Machine”. The tool backs up changes to the internal data storage in­cre­ment­ally to an external hard drive. As usual with in­cre­ment­al backups, an initial full backup is created during the first run. Sub­sequent changes to the file system are logged and are available for further backups.

Time Machine allows you to restore in­di­vidu­al files to earlier points in time. Fur­ther­more, the complete system can be re­con­struc­ted from the backup. This is practical in case of faulty hardware or if you want to migrate your own system to new hardware. Its simple handling of the backup software is par­tic­u­larly im­press­ive. The user only has to connect an external hard disk and launch the backup process; the rest is automated.

In­cre­ment­al backups are also available on Windows. On the one hand, a backup can be created in Windows 10 with the help of Windows Backup. On the other hand, you can use the Robocopy backup tool in the command line to in­cre­ment­ally backup the contents of a directory. Let’s look at an example:

The /MIR option here stands for `mirror´. The command mirrors the source directory to the des­tin­a­tion path. If a directory with the same name already exists, an in­cre­ment­al backup is performed. Robocopy then transfers only the changes as of the last backup operation.

Robocopy exists only under Windows. To create a server backup with Rsync under Linux, an in­cre­ment­al backup is used. First, a full backup is created. When the command is next called up, Rsync transfers the blockwise changes only as of the last backup. The trans­ferred data is merged with the existing data set. Thus, the result of the backup operation is a synthetic full backup. Here is an example of a cor­res­pond­ing Rsync call: