Proxmox Storage Configuration: Local, NFS and iSCSI
Comprehensive guide to configuring storage in Proxmox VE including local directory, LVM, ZFS, NFS, and iSCSI storage types.
# Proxmox Storage Configuration: Local, NFS and iSCSI
Introduction
Storage is a critical component of any Proxmox VE deployment, determining performance, reliability, and scalability for virtual machines and containers. Proxmox supports multiple storage types including local directories, LVM, LVM-Thin, ZFS, NFS, iSCSI, Ceph, and more.
This tutorial provides a comprehensive guide to Proxmox storage configuration, covering local storage setup, NFS network storage, iSCSI block storage, ZFS filesystems, and storage best practices. You will learn how to add, configure, and optimize storage for different workloads.
By the end of this guide, you will be able to configure Proxmox storage for VM disks, container root filesystems, backups, and ISO images.
Technical Background
Proxmox Storage Content Types
| Content Type | Description | |--------------|-------------| | images | Virtual machine disk images (QCOW2, RAW) | | rootdir | Container root directories (LXC) | | vztmpl | Container templates | | iso | Installation ISO images | | backup | Backup files (vma, tar, zst) | | snippets | Proxmox Backup Server snippets |
Storage Types Comparison
| Type | Protocol | Best For | Performance | |------|----------|----------|-------------| | Directory | Local filesystem | ISOs, templates, small VMs | Medium | | LVM | Block device | VM disks | High | | LVM-Thin | Thin-provisioned block | VM disks, overcommitment | High | | ZFS | Filesystem + volume manager | Production VMs, snapshots | High | | NFS | Network file system | Shared storage, backups | Medium | | iSCSI | Block over IP | SAN integration | High | | Ceph | Distributed storage | Clusters, HA | High |
ZFS RAID Levels
| RAID Level | Minimum Disks | Fault Tolerance | Capacity | |------------|---------------|-----------------|----------| | stripe | 1 | 0 | 100% | | mirror | 2 | 1 disk | 50% | | raidz1 | 3 | 1 disk | (n-1)/n | | raidz2 | 4 | 2 disks | (n-2)/n | | raidz3 | 5 | 3 disks | (n-3)/n |
Prerequisites
- Proxmox VE installed and operational
- Additional disks for storage (for LVM/ZFS/iSCSI)
- NFS server or iSCSI target (for network storage)
- Network connectivity between Proxmox and storage server
- Root access to Proxmox host
Architecture
Storage Layout Example
Proxmox Host
├── local (directory)
│ ├── /var/lib/vz/template/cache (container templates)
│ ├── /var/lib/vz/template/iso (installation ISOs)
│ └── /var/lib/vz/dump (backups)
├── local-lvm (LVM-Thin)
│ └── VM disk images (high performance)
├── nfs-backup (NFS)
│ └── Backup repository
└── iscsi-vmfs (iSCSI)
└── Shared VM storage (cluster)Step-by-Step Implementation
Step 1: List Available Disks
**Purpose**: Identify physical disks available for storage configuration.
**Commands**:
# List all block devices
lsblk
# List disks with details
fdisk -l
# Check existing Proxmox storage
pvesm status**Expected output**: List of physical disks (sda, sdb, sdc, etc.) with sizes.
**Verification**:
lsblk
# Shows disk devices and partitionsStep 2: Configure Local Directory Storage
**Purpose**: Set up directory-based storage for ISOs, templates, and backups.
**Commands**:
# Create directory for backups
mkdir -p /mnt/backups
# Add directory storage to Proxmox
pvesm add dir local-backup --path /mnt/backups --content backup,vztmpl,iso
# Verify storage added
pvesm status**Configuration files**: `/etc/pve/storage.cfg`
**Explanation**: Directory storage stores files as regular files on a filesystem. Suitable for ISOs, templates, and backups, but not recommended for high-performance VM disks.
**Expected output**: `local-backup` storage appears in `pvesm status` output.
**Verification**:
pvesm status
# Shows local-backup as active storageStep 3: Configure LVM-Thin Storage
**Purpose**: Create thin-provisioned block storage for VM disks.
**Commands**:
# Create LVM volume group (if disk is empty)
# Assuming /dev/sdb is available
vgcreate pve-data /dev/sdb
# Create thin pool
lvcreate -L 100G -T pve-data/thinpool
# Add to Proxmox
pvesm add lvmthin local-lvm --pool thinpool --vgname pve-data --content images,rootdir
# Verify
pvesm status**Explanation**: LVM-Thin allows overprovisioning (allocating more virtual disk space than physical space available). Use with monitoring to avoid running out of physical space.
**Expected output**: `local-lvm` storage active with thin pool.
**Verification**:
pvesm status
vgs
lvsStep 4: Configure ZFS Storage
**Purpose**: Set up ZFS filesystem with RAID-Z for production VM storage.
**Commands**:
# Create ZFS pool (mirror example with 2 disks)
zpool create zp0 mirror /dev/sdb /dev/sdc
# Create ZFS dataset for Proxmox
zfs create zp0/vm-data
zfs set mountpoint=/mnt/zp0 vm-data
# Add to Proxmox
pvesm add zfspool zp0-vm-data --pool zp0/vm-data --content images,rootdir
# Enable compression (recommended)
zfs set compression=lz4 zp0/vm-data
zfs set atime=off zp0/vm-data**Explanation**: ZFS provides copy-on-write snapshots, data checksumming, compression, and RAID-Z redundancy. Mirror configuration tolerates one disk failure.
**Expected output**: ZFS pool created and added to Proxmox storage.
**Verification**:
zpool status
pvesm status
zfs listStep 5: Configure NFS Storage
**Purpose**: Add network-attached storage via NFS for shared VM storage or backups.
**Commands** (on Proxmox host):
# Install NFS client (usually pre-installed)
apt install nfs-common
# Create mount point
mkdir -p /mnt/nfs-storage
# Test NFS server connectivity
showmount -e 192.168.1.50
# Add NFS storage to Proxmox
pvesm add nfs nfs-vm-storage --server 192.168.1.50 --export /mnt/vm-storage --content images,rootdir,backup
# Verify
pvesm status**Configuration files**: `/etc/pve/storage.cfg` (NFS mount configured automatically)
**Explanation**: NFS provides shared storage accessible by multiple Proxmox nodes. Suitable for clusters with shared VM storage or centralized backup repositories.
**Expected output**: NFS storage mounted and active in Proxmox.
**Verification**:
pvesm status
df -h | grep nfs
showmount -e 192.168.1.50Step 6: Configure iSCSI Storage
**Purpose**: Add block-level storage via iSCSI for SAN integration.
**Commands** (on Proxmox host):
# Install iSCSI initiator
apt install open-iscsi
# Discover iSCSI targets
iscsiadm -m discovery -t sendtargets -p 192.168.1.60
# Login to target
iscsiadm -m node -T iqn.2024-01.com.storage:target1 -p 192.168.1.60 --login
# Verify device appeared
lsblk | grep sd
# Add iSCSI storage to Proxmox
pvesm add iscsi iscsi-vm-storage --portal 192.168.1.60 --target iqn.2024-01.com.storage:target1 --content images
# Verify
pvesm status**Explanation**: iSCSI presents block devices over IP network. Proxmox can use iSCSI LUNs directly as VM disk storage. Requires initiator name configuration and CHAP authentication if required by target.
**Expected output**: iSCSI LUN available as Proxmox storage.
**Verification**:
pvesm status
iscsiadm -m session
lsblkConfiguration Examples
Complete Storage Configuration (`/etc/pve/storage.cfg`)
dir: local
path /var/lib/vz
content iso,vztmpl,backup,images
dir: local-backup
path /mnt/backups
content backup,iso,vztmpl
lvmthin: local-lvm
thinpool thinpool
vgname pve-data
content images,rootdir
zfspool: zp0-vm-data
pool zp0/vm-data
content images,rootdir
nfs: nfs-vm-storage
server 192.168.1.50
export /mnt/vm-storage
content images,rootdir,backup
options nfsvers=4.2
iscsi: iscsi-vm-storage
portal 192.168.1.60
target iqn.2024-01.com.storage:target1
content imagesNFS Server Export Configuration (`/etc/exports` on NFS server)
/mnt/vm-storage 192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)
/mnt/backups 192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)iSCSI Target Configuration Example (Linux target)
# /etc/target/saveconfig.json (targetcli-generated)
{
"iqn.2024-01.com.storage:target1": {
"lun0": {
"path": "/dev/sdb",
"type": "block"
},
"acls": ["iqn.2024-01.proxmox:pve01"]
}
}ZFS Optimization for Proxmox
zfs set compression=lz4 zp0/vm-data
zfs set atime=off zp0/vm-data
zfs set xattr=sa zp0/vm-data
zfs set recordsize=16K zp0/vm-data # For VM workloadsVerification
Storage Status Commands
pvesm status -- List all storage pools
pvesm path <storage> -- Show storage path
pvesm scan <storage> -- Scan storage contents
lsblk -- List block devices
zpool status -- ZFS pool status
df -h -- Filesystem usageExpected Output
Name Type Status Total Used
local directory active 200.00 GiB 50.00 GiB
local-lvm lvmthin active 500.00 GiB 100.00 GiB
nfs-vm-storage nfs active 2.00 TiB 500.00 GiB
iscsi-vm-storage iscsi active 1.00 TiB 200.00 GiB
local-backup directory active 4.00 TiB 1.00 TiBStorage Content Verification
# List ISO images in local storage
pvesm list local --content iso
# List container templates
pvesm list local --content vztmpl
# List backups
pvesm list local-backup --content backupTroubleshooting
| Symptom | Possible Cause | Diagnostic Command | Corrective Action | Verification | |---------|---------------|-------------------|-------------------|--------------| | Storage not showing | Configuration error | `pvesm status` | Check `/etc/pve/storage.cfg` | Storage appears | | NFS mount failed | Server unreachable or export wrong | `showmount -e <server>` | Fix export or network | NFS mounts | | iSCSI not connecting | Target not available or auth issue | `iscsiadm -m session` | Check target; verify CHAP auth | Session logged in | | ZFS pool degraded | Disk failure | `zpool status` | Replace disk; zpool replace | Pool ONLINE | | LVM-Thin overprovisioned | More virtual than physical space | `vgs` | Add disk or reduce allocations | Free space available | | Slow storage performance | Wrong storage type or disk type | `pveperf` | Use SSD/NVMe for VM disks | Performance improves | | Cannot create VM disk | Storage lacks 'images' content | `pvesm status` | Add 'images' to content types | VM disk created | | Backup storage full | Too many backups | `df -h` | Retain fewer backups; add storage | Free space available | | NFS stale file handle | Server reboot or export change | `dmesg \| grep nfs` | Unmount and remount NFS | Mount restored | | iSCSI path disappears | Network blip | `iscsiadm -m session` | Re-login to target | Path restored |
Best Practices
1. **Use SSDs/NVMe for VM disks**: Provides best performance for database and I/O-intensive workloads. 2. **Use ZFS for production**: Offers data integrity, snapshots, and self-healing capabilities. 3. **Separate backup storage**: Keep backups on different physical disks or systems from VM storage. 4. **Monitor thin pool usage**: LVM-Thin can run out of physical space if overprovisioned. 5. **Enable ZFS compression**: LZ4 compression reduces space usage with minimal CPU overhead. 6. **Use NFS v4.2**: Better performance and features than NFS v3 or v4.0. 7. **Configure CHAP for iSCSI**: Authenticate iSCSI connections for security. 8. **Test storage failover**: Verify storage access after network or disk failures. 9. **Monitor IOPS and latency**: Track storage performance metrics for capacity planning. 10. **Document storage layout**: Record all storage pools, content types, and purposes.
Conclusion
Proxmox VE storage configuration offers flexibility to match storage to workload requirements. Local directory storage is simple for ISOs and backups, LVM-Thin provides high-performance thin-provisioned block storage, ZFS delivers production-grade features with data integrity, and NFS/iSCSI enable networked shared storage for clusters. Proper storage selection and configuration directly impact VM performance, data safety, and operational simplicity. Monitor storage health regularly and plan capacity ahead of demand.
References
- Proxmox VE Storage Documentation (https://pve.proxmox.com/wiki/Storage)
- Proxmox VE ZFS (https://pve.proxmox.com/wiki/ZFS)
- Proxmox VE NFS (https://pve.proxmox.com/wiki/NFS)
- Proxmox VE iSCSI (https://pve.proxmox.com/wiki/ISCSI)
- ZFS on Linux Documentation (https://openzfs.github.io/openzfs-docs/)
- LVM2 Documentation (https://sourceware.org/lvm2/)
- NFS Documentation (https://nfs.sourceforge.net/)