Complete Guide to SAN NAS and DAS Storage Architectures
Deep dive into Storage Area Network, Network Attached Storage, and Direct Attached Storage architectures, use cases, and implementation.
# Complete Guide to SAN, NAS and DAS Storage Architectures
Introduction
Storage architecture is a foundational decision in enterprise infrastructure. The three primary block and file storage architectures — Direct Attached Storage (DAS), Network Attached Storage (NAS), and Storage Area Network (SAN) — each serve distinct workloads with different performance, scalability, and cost characteristics. Choosing the wrong architecture can lead to performance bottlenecks, capacity constraints, or unnecessary expense.
This guide provides a comprehensive technical comparison of DAS, NAS, and SAN, covering architecture, protocols, use cases, and implementation considerations.
You will learn:
- Architecture and components of DAS, NAS, and SAN
- Protocol differences (iSCSI, FC, FCoE, NFS, SMB)
- Performance, scalability, and cost characteristics
- Use case selection criteria
- Hybrid and converged storage models
- Implementation best practices
Technical Background
Direct Attached Storage (DAS)
DAS is storage directly connected to a single server or workstation. The storage is not shared with other systems over a network.
**Components:**
- Internal disk drives (SAS, SATA, NVMe)
- External storage enclosures connected via SAS, SATA, or USB
- RAID controller or HBA
- Storage controller integrated into server or external enclosure
**Characteristics:**
- Single-server access only
- No network protocol overhead
- Lowest latency for local workloads
- Limited scalability (bounded by server capacity)
- Simple management (no network storage configuration)
**Pros:**
- Lowest cost for small deployments
- Simplest setup and management
- Highest performance for local I/O
- No network dependency
**Cons:**
- Not shareable between servers
- Limited scalability
- Storage silos (underutilized capacity)
- No centralized backup point
Network Attached Storage (NAS)
NAS is a file-level storage device connected to a network, providing shared file access to multiple clients.
**Components:**
- Dedicated NAS appliance or server
- File system (ext4, XFS, ZFS, WAFL)
- Network interface (1GbE, 10GbE, 25GbE)
- File-sharing protocols (NFS, SMB/CIFS, FTP, SFTP)
- RAID controller or software RAID
**Characteristics:**
- File-level access (not block-level)
- Shared access for multiple clients
- Network-dependent performance
- Highly scalable (add NAS nodes or expand capacity)
- Centralized management
**Pros:**
- Easy file sharing across multiple systems
- Cross-platform support (NFS for Linux, SMB for Windows)
- Centralized backup and management
- Scalable capacity
- Lower cost than SAN for file workloads
**Cons:**
- Higher latency than DAS for block workloads
- Network dependency
- Not suitable for block-level applications (databases generally)
- Single point of failure (unless clustered)
Storage Area Network (SAN)
SAN is a dedicated high-speed network that provides block-level storage access to servers.
**Components:**
- Storage arrays (disk enclosures with controllers)
- Fibre Channel switches or Ethernet switches (for iSCSI/FCoE)
- Host Bus Adapters (HBA) or converged network adapters (CNA)
- Storage controllers with cache
- Zoning and LUN masking configuration
**Characteristics:**
- Block-level access (appears as local disk to server)
- Dedicated network separate from LAN
- High performance and low latency
- High scalability (petabytes, thousands of LUNs)
- Complex zoning and LUN management
**Pros:**
- Highest performance for block workloads
- Shared block storage for clusters
- Offloads storage traffic from LAN
- Mature enterprise ecosystem
- Supports clustering, live migration, replication
**Cons:**
- Highest cost (specialized hardware, fabric)
- Complex management (zoning, LUN masking, multipathing)
- Requires specialized skills (FC SAN expertise)
- Overkill for small deployments
Architecture Comparison
| Feature | DAS | NAS | SAN | |---------|-----|-----|-----| | Access Level | Block (local) | File | Block (network) | | Sharing | Single server | Multiple clients | Multiple servers | | Network | None | LAN (Ethernet) | Dedicated (FC or IP) | | Protocol | SATA/SAS/PCIe | NFS, SMB, FTP | FC, iSCSI, FCoE | | Latency | Lowest | Moderate | Low | | Scalability | Limited | High | Very High | | Cost | Low | Moderate | High | | Complexity | Low | Low | High | | Best Use | Single server, boot | File shares, home dirs | Databases, virtualization |
Protocol Deep Dive
Fibre Channel (FC)
FC is a dedicated high-speed network technology for storage.
**FC Components:**
- FC HBA (Host Bus Adapter) in server
- FC switches (fabric)
- FC storage ports on array
- FC cables (optical, copper)
**FC Topologies:**
- **Point-to-Point:** Direct server-to-storage (limited)
- **Arbitrated Loop:** Shared loop (legacy, limited performance)
- **Switched Fabric:** Multiple servers and storage via switches (enterprise standard)
**FC Speeds:** 1, 2, 4, 8, 16, 32, 64, 128 Gb/s
**Zoning:** Logical grouping of ports to control access. Types:
- Hard Zoning: Port-based, enforced by switch
- Soft Zoning: WWN-based, enforced by name server
iSCSI (Internet Small Computer System Interface)
iSCSI encapsulates SCSI commands over IP networks.
**Components:**
- iSCSI initiator (server software or hardware)
- iSCSI target (storage array or gateway)
- Standard Ethernet network
- CHAP authentication
**iSCSI Advantages:**
- Uses existing Ethernet infrastructure
- No specialized FC hardware required
- Can route over WAN
- Familiar to network administrators
**iSCSI Considerations:**
- TCP/IP overhead reduces efficiency
- Jumbo frames (9000 MTU) recommended for performance
- NIC offload (iSCSI HBA) reduces CPU overhead
- Multipathing (MPIO) for redundancy
FCoE (Fibre Channel over Ethernet)
FCoE encapsulates FC frames over Ethernet.
**Components:**
- Converged Network Adapter (CNA)
- Data Center Bridging (DCB) switches
- Lossless Ethernet required
**Benefits:**
- Converged network (storage + data on same cable)
- Reduced cabling and hardware
- FC protocol familiarity
**Considerations:**
- Requires DCB-compliant switches
- Lossless Ethernet configuration (PFC, ETS)
- Limited distance compared to FC
NFS (Network File System)
NFS is a distributed file system protocol for Unix/Linux.
**Versions:**
- NFSv2: Legacy, 32-bit, limited
- NFSv3: Widely deployed, stateless, UDP/TCP
- NFSv4: Stateful, improved performance, security, firewall-friendly
**NFS v4 Features:**
- Single port (2049)
- Integrated ACLs
- Improved caching
- Better WAN performance
- Delegation (client-side caching)
SMB/CIFS (Server Message Block)
SMB is the Windows file sharing protocol.
**Versions:**
- SMB 1.0: Legacy, insecure, disabled by default
- SMB 2.0/2.1: Improved performance, reduced chattiness
- SMB 3.0: Encryption, multichannel, scale-out, RDMA
- SMB 3.1.1: Enhanced security, pre-authentication integrity
**SMB 3 Features:**
- Transparent encryption
- Multichannel (multiple network paths)
- SMB Direct (RDMA over SMB)
- Scale-out file server
Use Case Selection
| Workload | Recommended Architecture | Reason | |----------|------------------------|--------| | Operating system boot | DAS or SAN | Local or SAN boot | | Database (OLTP) | SAN (FC or iSCSI) | Block-level, high performance, clustering | | Database (OLAP/warehouse) | SAN or NAS | Large capacity, good throughput | | File shares / home directories | NAS | Easy file sharing, cross-platform | | Virtualization (VMFS/VVol) | SAN | Shared block storage for clusters | | Virtualization (NFS datastore) | NAS | Simpler than SAN for smaller deployments | | Backup repository | NAS or SAN | Large capacity, centralized | | Media streaming | NAS | File-level, sequential reads | | Big data / analytics | SAN or DAS | High throughput, local processing | | HPC scratch storage | DAS or parallel NAS | Maximum local performance |
Hybrid and Converged Models
Hyper-Converged Infrastructure (HCI)
HCI combines compute, storage, and networking in a single appliance.
**Architecture:**
- Server nodes with local storage
- Distributed storage software (vSAN, Nutanix AOS, Ceph)
- Storage pooled across nodes
- Managed as single resource
**Pros:**
- Simple deployment and scaling
- Single management interface
- Built-in data protection
- Linear scalability
**Cons:**
- Hardware coupling (compute and storage scale together)
- Potential for higher cost at small scale
- Vendor lock-in risk
Converged Infrastructure
Pre-integrated bundles of compute, storage, and network from a single vendor.
**Examples:**
- VxBlock (Dell EMC, VMware, Cisco)
- FlexPod (NetApp, Cisco)
- HPE Synergy
Implementation Best Practices
DAS Best Practices
1. **Use hardware RAID** — Protects against drive failure with battery-backed cache 2. **Separate boot and data** — RAID 1 for OS, RAID 5/6/10 for data 3. **Monitor drive health** — SMART data, controller alerts 4. **Plan capacity growth** — Leave expansion bays empty for future drives 5. **Use enterprise drives** — 24/7 rated, better error recovery
NAS Best Practices
1. **Use redundant network** — Multiple NICs, LACP or failover 2. **Enable jumbo frames** — 9000 MTU for better throughput 3. **Implement quotas** — Prevent capacity exhaustion 4. **Configure snapshots** — Point-in-time recovery 5. **Use replication** — Off-site disaster recovery 6. **Monitor inode usage** — File count can exhaust before capacity
SAN Best Practices
1. **Implement zoning** — Restrict server access to appropriate LUNs 2. **Use LUN masking** — Additional access control at storage array 3. **Configure multipathing** — Redundant paths (MPIO, DM-Multipath) 4. **Separate fabrics** — Two independent FC fabrics for redundancy 5. **Monitor buffer credits** — FC performance depends on buffer-to-buffer credits 6. **Document zoning** — Critical for troubleshooting and changes
Architecture Diagram
graph TD
subgraph "DAS - Direct Attached Storage"
Server1[Server]
Drives1[(Disk Drives)]
Server1 -->|SATA/SAS/PCIe| Drives1
end
subgraph "NAS - Network Attached Storage"
NAS[NAS Appliance]
FS[File System]
Shares[NFS / SMB Shares]
NAS --> FS --> Shares
end
subgraph "SAN - Storage Area Network"
Fabric[FC/IP Fabric]
Array[Storage Array]
LUNs[LUNs]
Array --> LUNs
Fabric --> LUNs
end
subgraph "Clients"
VM1[VM Host 1]
VM2[VM Host 2]
App1[App Server 1]
end
Server1 -->|Local only| Drives1
VM1 -.->|NFS/SMB| Shares
VM2 -.->|NFS/SMB| Shares
App1 -.->|Block (FC/iSCSI)| LUNs
VM1 -.->|Block (FC/iSCSI)| LUNsStep-by-Step Implementation
Step 1: Assess Requirements
**Questions to answer:**
- What workloads will use the storage? (database, file shares, virtualization, backup)
- What performance is required? (IOPS, throughput, latency)
- What capacity is needed now and in 3 years?
- What availability is required? (single server, cluster, multi-site)
- What is the budget?
- What skills does the team have? (FC SAN expertise vs. general IT)
**Output:** Requirements document with performance, capacity, availability, and budget targets.
Step 2: Select Architecture
**Decision Matrix:**
Single server, boot drive → DAS
File sharing, home directories → NAS
Database, virtualization cluster → SAN
Mixed workloads → Hybrid (DAS boot + SAN data + NAS files)
Small budget, limited skills → DAS or NAS
Enterprise, high performance → SAN**Verification:** Architecture decision documented with justification.
Step 3: Design the Solution
**Design Components:**
- Storage array/model selection
- Drive type and capacity (SAS, SATA, NVMe, HDD, SSD)
- RAID levels per workload
- Network design (FC fabric, iSCSI VLANs, NAS shares)
- Zoning and LUN masking plan
- Multipathing configuration
- Backup and replication design
**Verification:** Design reviewed and approved by stakeholders.
Step 4: Install Hardware
**Installation Steps:** 1. Rack mount storage array 2. Install HBAs in servers 3. Connect FC or Ethernet cables 4. Connect power (redundant PDUs) 5. Power on and verify hardware health
**Verification:** All hardware detected, no errors in management interface.
Step 5: Configure Storage
**SAN Configuration Example:**
# On storage array:
# 1. Create disk groups (RAID groups)
# 2. Create LUNs from disk groups
# 3. Configure zoning on FC switches
# 4. Configure LUN masking on array
# 5. Present LUNs to appropriate servers
# On server (Linux):
# 1. Install multipath-tools
# 2. Rescan for new LUNs
echo "- - -" > /sys/class/scsi_host/host0/scan
# 3. Configure multipath
multipath -ll
# 4. Create filesystem on multipath device
mkfs.ext4 /dev/mapper/mpath0
# 5. Mount and add to fstab**NAS Configuration Example:**
# Create ZFS pool
zpool create tank raidz2 /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf
# Create filesystem
zfs create tank/shares
zfs set mountpoint=/shares tank/shares
# Configure NFS export
zfs set sharenfs="rw=@10.0.1.0/24" tank/shares
# Or SMB share
zfs set sharesmb=on tank/sharesStep 6: Configure Multipathing
**Linux DM-Multipath:**
# Install
apt install multipath-tools # Debian/Ubuntu
yum install device-mapper-multipath # RHEL/CentOS
# Enable
mpathconf --enable --with_multipathd y
systemctl enable multipathd
systemctl start multipathd
# View paths
multipath -ll**Verification:** Multiple paths visible, failover tested.
Step 7: Validate and Benchmark
**Validation Steps:**
- [ ] All LUNs/shares accessible from servers
- [ ] Multipath failover tested (pull cable, verify I/O continues)
- [ ] Performance meets requirements (fio benchmark)
- [ ] Backup jobs configured and tested
- [ ] Alerting configured (capacity, drive failure, path failure)
**Benchmark Command:**
# Random write performance (database simulation)
fio --name=randwrite --rw=randwrite --bs=8K --size=50G --numjobs=8 --iodepth=32 --runtime=120 --group_reporting
# Sequential read (backup simulation)
fio --name=seqread --rw=read --bs=1M --size=100G --numjobs=4 --iodepth=16 --runtime=120 --group_reporting**Verification:** Benchmark results meet or exceed requirements.
Configuration Examples
SAN Zoning Example (Brocade FC Switch)
# Create zones
zonecreate "zone_server1_storage1", "10:00:00:00:c9:2e:40:10; 10:00:00:00:c9:2e:50:20"
# Create zone configuration
cfgcreate "SAN_CFG", "zone_server1_storage1"
# Enable configuration
cfgenable "SAN_CFG"
# Save configuration
cfgsaveiSCSI Initiator Configuration (Linux)
# Install open-iscsi
apt install open-iscsi
# Discover targets
iscsiadm -m discovery -t st -p 10.0.1.100
# Login to target
iscsiadm -m node -T iqn.2024-01.com.example:storage.target1 -p 10.0.1.100 -l
# Verify
iscsiadm -m session
lsblkNFS Export Configuration
# /etc/exports
/shares/data 10.0.1.0/24(rw,sync,no_subtree_check,no_root_sqush)
# Apply
exportfs -a
# Client mount
mount -t nfs 10.0.1.100:/shares/data /mnt/nfsSMB Share Configuration (Samba)
# /etc/samba/smb.conf
[data]
path = /shares/data
browseable = yes
read only = no
valid users = @storage-group
write list = @storage-group
vfs objects = acl_xattr
map acl inherit = yes
store dos attributes = yesVerification
Connectivity Verification
# iSCSI
iscsiadm -m session
# Should show logged-in sessions
# NFS
showmount -e 10.0.1.100
# Should show exported shares
# SAN
multipath -ll
# Should show multiple paths with status active
# Generic
lsblk
# Should show all block devicesPerformance Verification
Compare benchmark results against expected values:
| Interface | Sequential Read | Random Read IOPS | Latency | |-----------|----------------|------------------|---------| | SATA SSD | ~550 MB/s | ~90K | ~100 µs | | SAS SSD | ~1100 MB/s | ~180K | ~80 µs | | NVMe (PCIe 3.0) | ~3500 MB/s | ~750K | ~25 µs | | NVMe (PCIe 4.0) | ~7000 MB/s | ~1.5M | ~15 µs |
Failover Verification
# Test multipath failover
multipath -ll # Note path status
# Pull one FC cable or disable one switch port
multipath -ll # Verify remaining path active
# Reconnect and verify failbackTroubleshooting
| Scenario | Symptom | Possible Cause | Diagnostic | Corrective Action | Verification | |----------|---------|----------------|------------|-------------------|--------------| | LUN not visible | Disk not in `lsblk` | Zoning issue, LUN masking, HBA not configured | `lspci`, `ssacli`, array zoning | Fix zoning, LUN masking, rescan HBA | LUN visible in `lsblk` | | Slow storage performance | Below expected bandwidth | Wrong RAID level, cache disabled, network congestion | `fio`, SSA performance metrics | Enable write cache, adjust RAID, check network | Performance meets baseline | | Path failure | One path down in multipath | Cable, SFP, switch port, HBA port | `multipath -ll`, switch port status | Replace cable/SFP, enable port | All paths active | | NAS not accessible | Mount failed | Export not configured, firewall, wrong IP | `showmount -e`, check firewall | Fix exports, open ports, verify IP | Mount succeeds | | SAN fabric down | All storage inaccessible | Switch failure, zoning error | Switch status, zoning config | Fix switch, correct zoning | Storage accessible | | Drive failure | Degraded array | Physical drive failure | SSA drive status, LED amber | Replace drive, allow rebuild | Array status OK | | Capacity full | No space on shares | Quota reached, snapshot consumption | `df -h`, `zfs list` | Expand capacity, delete snapshots | Space available | | Permission denied | Cannot access share | Wrong permissions, ACL issue | `ls -la`, `getfacl` | Fix permissions, ACLs | Access granted | | Multipath not working | Single path only | HBA mode not enabled, wrong driver | `multipath -ll`, HBA config | Enable HBA mode, install driver | Multiple paths active |
Best Practices
1. **Match architecture to workload** — DAS for simple, NAS for files, SAN for block/database 2. **Use enterprise-grade drives** — 24/7 rated drives with appropriate endurance 3. **Implement redundancy** — Dual controllers, multipathing, redundant fabrics 4. **Separate storage network** — Keep SAN traffic off production LAN 5. **Monitor capacity proactively** — Alert at 70%, 80%, 90% capacity 6. **Test failover regularly** — Quarterly failover tests for multipathing and controllers 7. **Document zoning and LUN masking** — Critical for troubleshooting 8. **Keep firmware updated** — Drive, controller, and HBA firmware 9. **Use appropriate RAID** — RAID 10 for performance, RAID 6 for capacity 10. **Plan for growth** — Leave headroom in arrays and fabrics
Conclusion
DAS, NAS, and SAN each serve distinct roles in enterprise storage architecture. DAS provides simple, low-latency storage for single servers. NAS enables easy file sharing across multiple clients. SAN delivers high-performance block storage for databases and virtualization clusters. Modern environments often use all three, with hybrid and converged models simplifying management.
The key to successful storage architecture is matching the technology to the workload requirements, implementing redundancy appropriately, and maintaining comprehensive documentation.
References
- SNIA Storage Networking Industry Association: https://www.snia.org/
- T11 Fibre Channel Standards: https://www.t11.org/
- iSCSI Specification (RFC 3720): https://tools.ietf.org/html/rfc3720
- NFS v4 Specification (RFC 7530): https://tools.ietf.org/html/rfc7530
- SMB Protocol Specification: https://docs.microsoft.com/en-us/openspecs/windows_protocols/
- HPE Storage Architecture: https://www.hpe.com/info/storage-architecture
- Dell EMC Storage Architecture: https://www.dell.com/en-us/storage.htm