PC Backup Software and Data Deduplication (Part III)

Our discussion on data deduplication redefining the way PC backup software works continues.

The third in a series, this article explains how various factors define the success of a PC backup. We have already explained
the following:

- Target Vs Source-based Deduplication.
- Inline Vs Post-process Deduplication
- File vs Sub-file Level Deduplication

Here’s the last of the four factors listed in the first article.

Fixed-Length Blocks v/s Variable-Length Data Segments

Fixed-length block approach, as the name suggests, divides the files into fixed size length blocks and uses simple checksum (MD5/SHA etc.) based approach to find duplicates. Although it’s possible to look for repeated blocks, the approach provides very limited effectiveness. The reason is that the primary opportunity for data reduction is in finding duplicate blocks in two transmitted datasets that are made up mostly - but not completely - of the same data segments.

For example, similar data blocks may be present at different offsets in two different datasets. In other words the block boundary of similar data may be different. This is very common when some bytes are inserted in a file, and when the changed file processes again and divides into fixed-length blocks, all blocks appear to have changed. Therefore, two datasets with a small amount of difference are likely to have very few identical fixed length blocks.

Variable-Length Data Segment technology divides the data stream into variable length data segments using a methodology that can find the same block boundaries in different locations and contexts. This allows the boundaries to “float” within the data stream so that changes in one part of the dataset have little or no impact on the boundaries in other locations of the dataset.


Among the better products PC backup software that use Variable-Length Data Segment technology is Druva inSync. Get a free trial today at - http://www.druva.com/insync/pc-backup