Backup and DR
Incremental Backup Explained
Incremental backup is a data backup mechanism designed to reduce storage requirements, bandwidth load, and provide the necessary level of data consistency and availability.
It this guide, we look at what incremental backup is, how incremental backup works and helps save time and money on backup and disaster recovery routines.
What Is Incremental Backup?
An incremental backup is a type of backup that copies only data that was changed since the previous backup. Unlike a full backup where all dataset is copied to the backup storage with every backup job, the incremental backup allows you to perform a full backup only once in a while. Every next backup will include only files that were changed since the most recent backup.
That is how this backup type gets its name: each backup is an increment for a previous backup.
Further reading Incremental and Differential Backup Comparison
How Incremental Backup Works
To demonstrate the incremental backup process, let’s assume you have made a full backup and now you would like to to do a weekly backup of any added or changed files:
- Week 1 - first full backup
- Week 2 - added and changed data only since the Week 1 backup
- Week 3 - added and changed data only since the Week 2 backup
Incremental backups require less storage space and network load and can be run more often since there are usually a lot of small everyday changes in the dataset that should be backed up.
Forever Incremental Backup
A forever incremental backup implies that you want incremental backups to be done at regular intervals after the initial full backup was performed. It seems to be very helpful that you don’t need to worry about full backups thereafter.
However, with each next incremental backup, the chain of subsequent incremental backups becomes larger. As a result, it takes more time and computing capacity for backup software to analyze the full backup and all increments to determine the difference between data on your server/workstation and the data in your backup repository.
Forever incremental backup decreases recovery reliability. It also becomes harder and takes longer to recover the whole dataset as it takes time to analyze and recover each incremental backup in the chain.
That is why periodical full backups are highly recommended to start the new sequence of incremental backups. The frequency of full backups depends on your business needs. You may want to conduct it weekly, monthly or once every couple of months. There is an advanced backup technique to simulate full backups called a synthetic full backup.
Further reading Synthetic full backup
Backup software vendors usually let users set the number of versions of data they would like to keep. Since the incremental backup implies the full backup and the chain of subsequent increments, and in terms of versioning, every next backup (full or incremental) is a version.
To provide data consistency you cannot eliminate separate elements of this chain, only the entire series of the full backup and all increments before the next full backup. This is why we strongly recommend performing full backups periodically.
Types of Incremental Backup
There are different incremental backup types that are determined by the size of changes that the software adds to the repository at every next incremental backup. We will overview the incremental file-level backup and the incremental block-level backup in detail.
Incremental File-Level Backup
In a file-level incremental backup: if a file has been modified, it will be sent to the backup repository to create a new version of it. This backup type is simple to perform and works well for a small dataset. However, this method might take longer because if the changed files are large, the backup will also be large, which will extend the duration of the backup job and require more storage space.
Further reading File Backup Best Practices
Incremental Block-Level Backup
In a block-level approach: the backup software performs a more in-depth file analysis and copies only the modified portions of files making it much faster to upload than using a file-level backup.
Further reading Block-Level Backup Explained
Using an Incremental Backup
Compared to the full backup, an incremental backup uploads only changed and new files, which allows the reduction of:
- Needed storage space
- Bandwidth consumption
- Computing resources usage
Let’s consider several different scenarios to back up a 5 GB dataset with a 3% daily change rate for one month and see how effective they are and how much storage space you need.
- Weekly full backups only. In this case, you will need to set up only 4-5 backups a month which is quite storage-friendly, especially if your retention policy doesn’t require you to keep several versions of every backup. The main issue here is data reliability. If something malicious happens to your servers or endpoints, you will permanently lose about a week of critical business data. Storage space calculation: 5 * 5 GB = 25 GB
Further reading RTO vs. RPO: Two Means toward the Same End
- Full backups daily. When you set full backups every day, it is quite reliable, isn’t it? However, it is unlikely someone is going to use this scenario as it requires so much storage room, bandwidth, and computing capacity that this backup strategy becomes highly ineffective. Storage space calculation: 22 * 5 GB = 110 GB
- Weekly full backups with incremental backups on weekdays. In this scenario, full backups don’t have to be set very often, and the periods between full backups are covered by incremental backups. Therefore, this scenario turns out to be the golden mean between saving storage space and providing data reliability. Storage space calculation: (5 * 5 GB) + (22 * 3% * 5 GB) = 28.3 GB
An incremental backup approach has several drawbacks:
- Longer data restore. It takes more time for backup software to find, analyze and collect the latest full backup and all subsequent incremental backups.
- Less data consistency and reliability. As we have already mentioned, when you need to recover, the backup software analyzes the whole incremental backup chain. If one element in that chain is corrupt, your data is non-recoverable.
Incremental Backup Best Practices
To make your recovery from incremental backup easier, here are a few tips:
- Perform full backups from time to time. Starting a new full backup sequence is a good way to ensure you can recover data.
- Set a retention policy for your files. A retention policy allows you to choose how long to keep your old files and how many versions of each file you keep. With an incremental backup, it’s recommended to set a policy that allows you to delete old files after a given time, or files that have been deleted from your device.
Out of all backup types, the incremental backup is one of the most beneficial backup methods as it assures reliability and accessibility of your data, saves a lot of storage space, consumes less bandwidth, and leads to faster backups compared to a full backup scenario. The flaws of incremental backup such as slower restore and dependence on the increments integrity may be addressed by performing the full backup periodically and setting the proper retention policy for your backups.