A SAN is a key piece of technology in data storage in several types of infrastructure, but there are varying implementations and standards on the market. In this article, we are going to review the key differences between FC, FCoE and iSCSI in order to help you choose the best option.
First, let’s quickly discuss storage area network (SAN) basics — networks used by storage devices and their clients (application/database/web servers, etc). SAN operates with data blocks, so it is a lower-level solution than shared network storage. SAN is extremely durable and fast thanks to low data transfer latencies and a focus on guaranteed data block delivery.
If you are confused by the term “block-level access,” the closest comparable technology is a USB external drive. Your PC accesses it using block-level protocols via cables as a transport “network.”
These blocks are a portion of binary data allocated onto the storage. In the case of a USB drive, it is a simple data block. But when referring to SAN storage, these blocks are called “chunks,” as they can be allocated across an array of hard drives and vary in size.
Operating systems combine these blocks into a usable data volume, with file systems and files/folders.
SAN networks are built with fault-tolerance and performance in mind:
- Allowance for simple multipathing.
- Block-level data access and reliable transfers for use with high-availability clusters.
- High throughput allows for the implementation of highly loaded systems.
- Network functions mostly offloaded to hardware, thus enabling better server performance.
There are a few key SAN-related technologies available on the market: Fibre Channel (FC), FC over Ethernet (FCoE) and iSCSI. Below is an overview of each.
Fibre Channel stands for a set of protocols, technologies and services used to build a “classic” SAN network. Here are the main types and their differences:
- Fibre Channel Protocol (FCP) - data transfer protocol that lets through SCSI commands.
- Fibre optic infrastructure - used to transmit data to and/or from FC devices.
- Name Service - acts as a database for connected devices. It is quite similar to a domain name system (DNS).
- Set of flow control services.
The picture below illustrates a storage network separated from the main LAN:
In most cases, “FC SAN” implies a storage network built up of dedicated hardware adapters and switches, connected using fiber optics. As the network is developed for high-loaded storage devices, it uses a strong cyclic redundancy check (CRC) — a hash function used to produce a checksum in order to detect errors in data. This ensures the data is not corrupted when transmitted. The network also has fewer retransmissions compared to TCP/IP and connection retries due to the loss of data. Typical Fibre Channel speeds are 8Gbps, 16Gbps, and 32Gbps.
When further comparing FC with TCP/IP-based networks, you will find Fibre Channel is more isolated. User and client devices are not directly connected to storage networks. This lowers security risks and the risk of human error. In addition, malware from a regular LAN cannot impact FC SAN directly.
Since FC requires buying special network switches and storage adapters, you may consider an Ethernet-based network to be a more straightforward, better choice. We’ll explore this in the next section.
The basic concept of iSCSI is simply putting SCSI commands inside of a typical TCP/IP channel. If you have a regular LAN, you can even install and configure iSCSI Target/Initiator software onto your storage server and its clients, thus facilitating SAN. It allows the same block-level storage access but uses conventional Ethernet networks.
Since underlying networks (TCP/IP over Ethernet) are designed for common usage, there are no strong data flow controls or built-in storage discovery services — You need to configure almost everything yourself. For example, you need to manually configure the IP addresses of iSCSI storage and clients, frame sizes, LUN visibility, etc. You will also need to optimize the network for large data block transfers to get relatively high performance. Additionally, you will need to purchase hardware-accelerated network adapters to offload iSCSI processing from a host server or client.
If you’re considering high-loaded storage network implementation, you should consider implementing a dedicated 10Gbps Ethernet-based network (either optical or copper). It should include hardware-accelerated adapters with network switches supporting larger data frame transfers. In this case, however, you will lose the main benefit of iSCSI — support of existing network and hardware.
It’s also important to consider that performance issues with a poorly designed iSCSI network are very hard to investigate and overcome, as there are several “common” protocols and services involved.
You will also encounter hardware and network settings like Flow Control, Jumbo Frames, Spanning Tree, Trunks, etc. if you choose iSCSI for production.
FCoE vs. iSCSI
There is one more Ethernet-based SAN technology, combining the benefits of both aforementioned options — Fibre Channel over Ethernet. Unlike iSCSI, it does not use the TCP/IP stack and lets FC protocol run inside lossless Ethernet:
The main benefit of FCoE is its ability to use “converged” infrastructure, which allows LAN and SAN traffic to pass through using the same hardware and links, while still maintaining isolation. The best examples are “Blade” servers: a rack that packs a lot of special servers in one piece of computing with shared power, network, storage, and management. Using copper Ethernet lanes and switches with the support of FCoE, it allows system administrators to change LAN/SAN distribution depending on business needs, so the system’s scaling is relatively easy.
If you’re using high-performance networks (10Gbps or better) and FCoE-enabled switches and adapters, this technology is a good alternative to the classic Fibre Channel (almost the same advantages, with the addition of configuration flexibility through universal transmission layers).
Fibre Channel vs FCoE vs iSCSI: Which is Better for the SAN?
Here is a quick pros/cons chart comparing Fibre Channel vs FCoE vs iSCSI to help you make your decision:
|Allows for use on existing network||-||+/-|
(needs special switches and adapters)
|Strong data flow management|
(CRC check, avoids transmission retries)
|Embedded service infrastructure||+||+||-|
|Network isolation by design||+||+||-|
|Need to buy special hardware|
(adapters, switches, etc.)
(good performance impossible without special adapters)
|Transfer speeds available||2/4/8/16/32Gbps, support for link aggregation||1/10/40/100Gbps,|
support for link aggregation
support for link aggregation
If you’re running a small company infrastructure or testing environment, choose iSCSI using existing networks and adapters. Gigabit Ethernet is enough for use with non-mission-critical, more demanding applications. But remember to split the logical “storage network” and user LAN to improve performance. (We also suggest, at the very least, enabling Jumbo Frames on all devices involved in storage network transmissions.)
When considering storage implementation solutions for demanding apps, a rapidly growing environment, or use with cluster systems, choose Fibre Channel. This technology is well-known, and most of your concerns will be addressed. When compared to the same zero-point deployment of FCoE, hardware costs are similar.
If dealing with modern converged hardware (e.g. Blade servers), the best choice is FCoE, thanks to its flexibility.