Enterprise Backup Architecture Full Incremental and Differential Backups

# Enterprise Backup Architecture: Full, Incremental and Differential Backups

Introduction

Backup is the last line of defense against data loss from hardware failure, ransomware, human error, or site disasters. A well-designed backup architecture balances backup windows, recovery time objectives (RTO), recovery point objectives (RPO), storage costs, and operational complexity.

This guide covers backup types, architectures, retention policies, and implementation best practices.

You will learn:

• Backup types (full, incremental, differential, synthetic, forever incremental)
• Backup architectures (target-based, source-based, cloud)
• Backup targets (tape, disk, cloud, object storage)
• Deduplication and compression
• Retention policies and grandfather-father-son (GFS)
• Recovery procedures and testing
• Backup verification and monitoring

Technical Background

Backup Types

| Backup Type | What's Backed Up | Backup Time | Restore Time | Storage Used | |-------------|-----------------|-------------|--------------|--------------| | Full | All data | Slowest | Fastest | Highest | | Incremental | Changed since last backup | Fast | Slow (needs all incrementals) | Low | | Differential | Changed since last full | Moderate | Moderate | Moderate | | Synthetic Full | Full from previous full + incrementals | Fast (no production impact) | Fast | Same as full | | Forever Incremental | Initial full, then incrementals only | Fast | Fast (with synthetic full) | Low |

Full Backup

**How it works:** Copies all selected data regardless of when it last changed.

**Characteristics:**

• Simplest restore (single backup set)
• Longest backup window
• Highest storage consumption
• Resets archive bit on files

**Use Cases:**

• Weekly or monthly full backups
• Initial backup for new systems
• Before major changes (migrations, upgrades)

Incremental Backup

**How it works:** Copies only data changed since the last backup (full or incremental).

**Characteristics:**

• Shortest backup window
• Lowest storage consumption
• Complex restore (need full + all incrementals in order)
• Any incremental failure breaks the chain

**Use Cases:**

• Daily backups
• Environments with limited backup window
• High-change data with good change tracking

Differential Backup

**How it works:** Copies all data changed since the last full backup.

**Characteristics:**

• Moderate backup window
• Moderate storage consumption (grows between fulls)
• Simpler restore than incremental (need full + latest differential)
• More resilient than incremental (single point of failure is the full)

**Use Cases:**

• Daily backups with moderate change rates
• Environments wanting simpler restore than incremental

Synthetic Full Backup

**How it works:** Creates a full backup from previous full + incrementals without accessing production data.

**Characteristics:**

• No production impact (reads from backup storage)
• Fast full backup equivalent
• Simplifies restore (single backup set)
• Requires backup server with sufficient I/O

**Use Cases:**

• Weekly synthetic fulls to simplify restore chains
• Environments where production impact must be minimized

Forever Incremental Backup

**How it works:** One initial full backup, then only incrementals. Periodic synthetic fulls create restore points.

**Characteristics:**

• Minimal daily backup window
• Minimal daily storage consumption
• Restore requires synthetic full + incrementals
• Most efficient long-term

**Use Cases:**

• Large environments with daily backup constraints
• Cloud backup targets

Backup Architecture

Target-Based Backup

**How it works:** Backup server reads data from production servers and writes to backup target.

**Components:**

• Backup server (media server)
• Backup target (tape library, disk array, object storage)
• Agents on production servers (or agentless)

**Pros:**

• Centralized management
• Mature ecosystem
• Deduplication on backup server

**Cons:**

• Backup traffic on production network
• Backup server is bottleneck
• Agents on production servers

Source-Based Backup

**How it works:** Production servers send data directly to backup target.

**Pros:**

• No backup server bottleneck
• Reduced production network impact
• Scales with number of sources

**Cons:**

• More complex management
• Each source needs configuration
• Less centralized control

LAN-Free and Server-Free Backup

**LAN-Free:** Backup data goes over SAN directly to tape/disk, bypassing LAN. **Server-Free:** Third-party copy (storage array to backup target), no server involvement.

**Benefits:**

• No LAN impact
• Higher throughput
• Reduced server CPU usage

**Requirements:**

• SAN infrastructure
• Shared storage visibility
• Vendor-specific integration

Backup Targets

Tape (LTO)

**LTO Generations:**

| Generation | Native Capacity | Compressed Capacity | Max Transfer Rate | |------------|----------------|---------------------|-------------------| | LTO-8 | 12 TB | 30 TB | 900 MB/s | | LTO-9 | 18 TB | 45 TB | 1,000 MB/s | | LTO-10 | 36 TB | 90 TB | 1,600 MB/s (planned) |

**Pros:**

• Lowest cost per TB
• Long archival life (30 years)
• Offline (air-gapped) protection against ransomware
• Portable (offsite storage)

**Cons:**

• Sequential access (slow restore of individual files)
• Mechanical reliability concerns
• Requires tape library and drives

Disk (Deduplication Appliance)

**Examples:** Dell PowerProtect DD, HPE StoreOnce, Quantum DXi

**Pros:**

• Fast backup and restore
• Random access (file-level restore)
• Deduplication reduces storage
• Random access for instant recovery

**Cons:**

• Higher cost per TB than tape
• Power and cooling requirements
• Not air-gapped (vulnerable to ransomware)

Cloud Storage

**Examples:** AWS S3, Azure Blob, Google Cloud Storage, Backblaze B2

**Pros:**

• Offsite by default
• Scalable
• No hardware management
• Multiple storage tiers (hot, cool, archive)

**Cons:**

• Ongoing costs
• Egress fees for restore
• Internet dependency
• Data sovereignty concerns

Object Storage (On-Premises)

**Examples:** MinIO, Ceph, Dell EMC ECS

**Pros:**

• S3-compatible API
• Scalable
• Cost-effective
• Private cloud control

**Cons:**

• Requires hardware and management
• Less mature backup integration

Deduplication and Compression

Deduplication Types

**Source-Side Deduplication:**

• Dedupe before sending over network
• Reduces network bandwidth
• More CPU on source server

**Target-Side Deduplication:**

• Dedupe after receiving at backup target
• Simpler source servers
• More CPU/storage on target

**Inline vs Post-Process:**

• **Inline:** Dedupe during write (slower backup, less storage)
• **Post-Process:** Dedupe after write (faster backup, temporary full storage)

Compression

• Reduces data size before encryption/dedupe
• Typically 1.5x-2x for text, less for already-compressed data
• CPU overhead on source or target

Typical Deduplication Ratios

| Data Type | Dedup Ratio | |-----------|-------------| | Virtual machines | 20:1 - 50:1 | | File servers | 10:1 - 20:1 | | Databases | 3:1 - 10:1 | | Email | 5:1 - 10:1 | | Already compressed (images, video) | 1:1 - 2:1 |

Retention Policies

Grandfather-Father-Son (GFS)

**Structure:**

• Daily (Son): Kept 1-2 weeks
• Weekly (Father): Kept 4-12 weeks
• Monthly (Grandfather): Kept 12-60 months
• Yearly: Kept 1-7 years

**Example:**

Daily backups: Mon-Sun (kept 7 days)
Weekly backups: Saturday (kept 4 weeks)
Monthly backups: Last Saturday of month (kept 12 months)
Yearly backups: Last Saturday of December (kept 7 years)

Retention by Regulation

| Regulation | Retention Period | Data Type | |------------|-----------------|-----------| | GDPR | As long as necessary | Personal data | | HIPAA | 6 years | Medical records | | SOX | 7 years | Financial records | | PCI DSS | 1 year | Cardholder data | | SEC 17a-4 | 3-7 years | Financial communications | | MiFID II | 5 years | Trading records |

Legal Hold

• Override normal retention
• Preserve data for litigation
• Can span multiple years
• Requires coordination with legal team

Recovery Procedures

Recovery Point Objective (RPO)

Maximum acceptable data loss (measured in time).

**Example RPOs:**

• Critical database: 15 minutes
• Email: 4 hours
• File shares: 24 hours
• Archive: 1 week

Recovery Time Objective (RTO)

Maximum acceptable downtime.

**Example RTOs:**

• Critical application: 1 hour
• Email: 4 hours
• File server: 8 hours
• Archive: 24 hours

Restore Procedures

**Full Restore:** 1. Identify backup set 2. Restore full backup 3. Apply incrementals/differentials in order 4. Verify data integrity

**File-Level Restore:** 1. Browse backup catalog 2. Select files/folders 3. Restore to original or alternate location 4. Verify files

**Bare Metal Restore:** 1. Boot from recovery media 2. Restore OS and drivers 3. Restore data from backup 4. Verify system boots

Instant Recovery

**How it works:** Run VM directly from backup storage while restore happens in background.

**Benefits:**

• Near-zero RTO for VMs
• Application available during restore

**Requirements:**

• Backup solution with instant recovery
• Sufficient backup storage performance

Backup Verification

Backup Verification Methods

| Method | Description | Frequency | |--------|-------------|-----------| | Catalog verification | Check backup catalog integrity | Daily | | File-level restore test | Restore sample files | Weekly | | Application restore test | Restore and boot application | Monthly | | Full disaster recovery test | Full site recovery | Quarterly | | Bare metal restore test | Full server recovery | Quarterly |

Automated Verification

• Backup software verification jobs
• Checksum validation
• Backup application health checks
• Alert on backup failures

Recovery Testing

**Test Scenarios:** 1. Single file restore 2. Application server restore 3. Database point-in-time recovery 4. Full site disaster recovery 5. Ransomware recovery scenario

**Test Documentation:**

• Test date and scenario
• RPO/RTO achieved
• Issues encountered
• Lessons learned
• Sign-off

Backup Monitoring

Key Metrics

| Metric | Description | Alert Threshold | |--------|-------------|-----------------| | Backup success rate | % of successful backups | < 99% | | Backup duration | Time to complete backup | > backup window | | Backup size | Data backed up daily | Rapid growth | | Deduplication ratio | Space savings | Sudden drop | | Tape/disk utilization | Target capacity | > 85% | | Restore success rate | % of successful restores | < 100% |

Alerting

• Backup job failures
• Backup window overruns
• Capacity thresholds
• Media errors
• Offsite replication failures

Architecture Diagram

graph TD
    subgraph "Production Environment"
        DB1[Database Server]
        FS1[File Server]
        VM1[VM Host]
        App1[Application Server]
    end

    subgraph "Backup Network"
        BackupNet[Backup VLAN]
    end

    subgraph "Backup Infrastructure"
        BS[Backup Server]
        BD[Backup Deduplication Appliance]
        TL[Tape Library]
    end

    subgraph "Offsite"
        Cloud[Cloud Storage]
        OffsiteTape[Offsite Tape Storage]
    end

    DB1 -->|LAN-Free/SAN| BD
    FS1 -->|LAN| BS --> BD
    VM1 -->|VMware API| BS --> BD
    App1 -->|Agent| BS --> BD

    BD -->|Replication| Cloud
    BD -->|Archive| TL
    TL -->|Transport| OffsiteTape

Step-by-Step Implementation

Step 1: Define Requirements

**Determine:**

• RPO for each system (how much data loss is acceptable)
• RTO for each system (how long can it be down)
• Retention requirements (regulatory, business)
• Backup window (when can backups run)
• Storage requirements (how much data, growth rate)

Step 2: Select Backup Solution

**Options:**

• Enterprise: Veeam, Commvault, NetBackup, Spectrum Protect
• Mid-market: Veeam, HPE StoreOnce, Dell PowerProtect
• Open source: Bacula, BareOS, rsnapshot
• Cloud-native: AWS Backup, Azure Backup, Google Cloud Backup

Step 3: Design Backup Architecture

**Components:**

• Backup server/media server
• Backup target (disk, tape, cloud)
• Agents (or agentless)
• Network (dedicated backup VLAN)
• Offsite copy

Step 4: Configure Backup Jobs

**Job Types:**

• Full backup (weekly)
• Incremental/Differential (daily)
• Transaction log backup (hourly for databases)
• System state/bare metal (weekly)

Step 5: Configure Retention

**GFS Example:**

• Daily: 7 days
• Weekly: 4 weeks
• Monthly: 12 months
• Yearly: 7 years

Step 6: Configure Offsite Copy

**Options:**

• Tape rotation to offsite
• Replication to cloud
• Replication to secondary site

Step 7: Test Restores

**Schedule:**

• Weekly: File-level restore test
• Monthly: Application restore test
• Quarterly: Full disaster recovery test

Step 8: Monitor and Optimize

**Monitor:**

• Backup success/failure
• Backup duration
• Deduplication ratio
• Target capacity
• Restore success rate

Configuration Examples

Veeam Backup Job (PowerShell)

# Create backup job
Add-VBRJob -Name "Daily_Servers" -Description "Daily incremental backup" -Schedule "[Daily] 22:00"

# Add objects
$servers = Get-VBRServer | Where-Object {$_.Name -like "SRV-*"}
Add-VBRJobObject -Job (Get-VBRJob -Name "Daily_Servers") -Server $servers

# Configure retention
$job = Get-VBRJob -Name "Daily_Servers"
$job.Options.BackupStorageOptions.RetainDailyCycles = 7
$job.Options.BackupStorageOptions.RetainWeeklyCycles = 4
$job.Options.BackupStorageOptions.RetainMonthlyCycles = 12
$job.Options.BackupStorageOptions.RetainYearlyCycles = 7
$job.Options.BackupStorageOptions.RepeatFullAlways = $true
$job.Options.BackupStorageOptions.FullBackupFrequency = Weekly
$job.Options.BackupStorageOptions.FullBackupDay = Saturday
$job.SetOptions($job.Options)

rsnapshot (Linux)

# /etc/rsnapshot.conf
config_version  1.2
snapshot_root   /var/snapshot/
no_create_root  1
cmd_cp          /bin/cp
cmd_rm          /bin/rm
cmd_rsync       /usr/bin/rsync
cmd_ssh         /usr/bin/ssh
cmd_logger      /usr/bin/logger
cmd_du          /usr/bin/du
cmd_rsnapshot_diff /usr/bin/rsnapshot-diff
interval        daily  7
interval        weekly 4
interval        monthly 1
backup          /home/ localhost/
backup          /etc/ localhost/
backup          /var/www/ localhost/

# Run via cron
# /etc/cron.d/rsnapshot
0 4 * * * root /usr/bin/rsnapshot daily
0 3 * * 0 root /usr/bin/rsnapshot weekly
0 2 1 * * root /usr/bin/rsnapshot monthly

Bareos Director Configuration

# /etc/bareos/bareos-dir.d/director.conf
Director {
  Name = bareos-dir
  Password = "changeme"
  QueryFile = "/etc/bareos/query.sql"
  Maximum Concurrent Jobs = 1
}

# /etc/bareos/bareos-dir.d/client/myserver-fd.conf
Client {
  Name = myserver-fd
  Address = myserver
  Password = "changeme"
  File Retention = 30 days
  Job Retention = 6 months
  AutoPrune = yes
}

Verification

Backup Job Verification

# Check backup logs
tail -f /var/log/backup.log

# Verify backup catalog
# Check backup server management interface

# Verify restore capability
# Perform test restore weekly

Restore Test Procedures

**File Restore Test:** 1. Select random file from backup 2. Restore to alternate location 3. Compare with original (checksum) 4. Document result

**Application Restore Test:** 1. Stop application 2. Restore from backup 3. Start application 4. Verify functionality 5. Document RTO achieved

Troubleshooting

| Scenario | Symptom | Possible Cause | Diagnostic | Corrective Action | Verification | |----------|---------|----------------|------------|-------------------|--------------| | Backup job fails | Error in backup logs | Connectivity, credentials, disk full | Check logs, check target capacity | Fix connectivity, credentials, capacity | Backup succeeds | | Backup window exceeded | Backup still running at start of day | Slow target, too much data, network | Check backup duration, target performance | Add backup window, upgrade target, dedup | Backup completes in window | | Restore fails | Cannot restore from backup | Corrupted backup, missing media | Check backup integrity, media inventory | Repair backup, locate media | Restore succeeds | | Tape errors | Backup fails on tape | Dirty head, worn tape, drive failure | Clean drive, check tape health | Clean drive, replace tape/drive | Backup to tape succeeds | | Dedup ratio drops | Sudden storage growth | New data type, dedup disabled | Check dedup stats | Investigate data, re-enable dedup | Ratio restored | | Offsite copy fails | Replication error | Network, credentials, target | Check replication logs | Fix network, credentials | Replication succeeds | | Backup agent offline | Agent not communicating | Agent stopped, network issue | Check agent service, network | Restart agent, fix network | Agent online | | Insufficient storage | Backup target full | Retention too long, data growth | Check target capacity | Add storage, adjust retention | Capacity available |

Best Practices

1. **3-2-1 Rule:** 3 copies, 2 different media, 1 offsite 2. **Test restores regularly** — A backup is only good if you can restore 3. **Monitor backup jobs** — Alert on failures immediately 4. **Use encryption** — Encrypt backups at rest and in transit 5. **Implement immutability** — Protect against ransomware (WORM, object lock) 6. **Separate backup network** — Don't impact production traffic 7. **Document procedures** — Runbooks for restore scenarios 8. **Test full disaster recovery** — Quarterly DR tests 9. **Keep backup software updated** — Security patches and features 10. **Document RPO/RTO** — Ensure they meet business requirements

Conclusion

A robust backup architecture is essential for data protection. The right combination of backup types, targets, and retention policies depends on your RPO/RTO requirements, data types, and budget. Regular testing and monitoring ensure backups are reliable when needed.

Remember: backup is not complete until a restore has been successfully tested.

References

• NIST SP 800-34 (Contingency Planning): https://csrc.nist.gov/publications
• SNIA Backup and Recovery: https://www.snia.org/
• Veeam Backup & Replication: https://www.veeam.com/
• Dell PowerProtect DD: https://www.dell.com/en-us/data-protection/
• HPE StoreOnce: https://www.hpe.com/info/storeonce
• AWS Backup: https://aws.amazon.com/backup/