What Is NoF+?
In the all-flash era, conventional Fiber Channel (FC) storage networks cannot meet the requirements of all-flash data centers (DCs). In response, the industry developed a Non-Volatile Memory Express (NVMe) over Fabric (NOF) storage network. This network is designed based on the emerging NVMe storage protocol, which significantly improves storage throughput performance and transmission delay in storage systems. There are several fabric technologies; however, storage vendors tend to prefer NVMe over RDMA over Converged Ethernet (RoCE), which becomes the mainstream of NoF in the industry. Within this context, Huawei launched the NoF+ Storage Network Solution optimized for all-flash, as it drastically improves performance, reliability, and ease-of-use compared with standard NoF.
Why Do We Need NoF+?
As we enter the intelligence era, there are more and more services with massive data storage and read/write requirements. This drives the emergence of new storage media (SSD) and storage protocols (NVMe), which in turn leads to 100-fold improvements in storage performance. Storage nodes are interconnected over a storage network. Improved storage performance calls for a faster and higher-quality storage network. For this, NVMe over Fabric, also called NoF, is introduced.
NoF applies the NVMe protocol to the front-end server host to connect the storage array with the front-end host. The NVMe protocol can replace the SCSI protocol in the SAN to construct an all-Ethernet SAN, as shown in Figure 1-2.
NVMe can be carried over a Remote Direct Memory Access over Converged Ethernet (RoCE), Fibre Channel (FC), or TCP network.
- NVMe over FC (FC-NVMe) is similar to FC-SCSI in that it is based on FCP and its I/O interactions are based on exchanges. This means that FC-NVMe and FC-SCSI can run on the same FC network by upgrading the host driver and switches. FC-NVMe reuses the infrastructure of FC networks to carry the NVMe protocol, thereby improving transmission.
- NVMe over TCP is based on the existing IP networks and uses the TCP protocol to transmit NVMe data. It implements end-to-end NVMe without changing the network infrastructure. Although NVMe over TCP does not deliver as good network performance as NVMe over FC or NVMe over RoCE, its overall performance is much better than that of Internet Small Computer Systems Interface (iSCSI). In addition, NVMe over TCP has low requirements on networks and better compatibility. For example, it can even run on conventional Ethernet, avoiding the need to build a lossless network. Therefore, in future markets, NVMe over TCP will be a common choice when high performance is not a priority.
- NVMe over RoCE is a type of NVMe over RDMA. RDMA is a native network protocol that carries NoF. In addition to RoCE, RDMA includes InfiniBand (IB) and Internet Wide Area RDMA Protocol (iWARP). Ethernet-based RoCE has become the mainstream network bearer mode of RDMA. NVMe over RDMA is relatively simple — it maps the I/O queues of the NVMe device to the RDMA Queue Pairs (QPs), and completes I/O interactions using the RDMA Send, RDMA Write, and RDMA Read semantics. NVMe over RoCE carries the NVMe protocol using the RDMA technology of the converged Ethernet.
Ethernet-based RoCE offers higher performance (higher bandwidth and lower delay) than FC and has TCP advantages (all-Ethernet and all-IP). All this makes NVMe over RoCE the optimal bearer network solution for NoF, and the mainstream technology of NoF in the industry.
RoCE has significant advantages over FC in terms of storage performance and bandwidth. However, when replacing FC and connecting to all-flash storage, there remains three improvements needed to standard NVMe over RoCE:
- Network performance: zero packet loss
Zero packet loss is a basic requirement for storage networks. However, traditional Ethernets are prone to packet loss upon congestion.
- Reliability: active/standby switchover within seconds
Multiple network planes are constructed to ensure storage reliability; an active/standby switchover needs to take less than 1s.
- Usability: plug-and-play
An FC storage network is simple and easy to configure. But some aspects of the current Ethernet need to be improved to better fit storage scenarios.
What Are the Advantages of NoF+ over NoF?
Building on the mainstream NoF solution, Huawei has improved network performance, reliability, and usability, drawing on its extensive experience in networking and storage. Based on the intelligent lossless network, Huawei launched the NoF+ Solution for centralized storage, expanding the use of DC storage networks.
- Enhanced network performance
Huawei's NoF+ Solution replaces static thresholds used in traditional Ethernet and optimizes network prediction capability. It calculates samples and uses Huawei's unique iLossless algorithm to accurately control thresholds in specific scenarios. In this way, it can predict network service requirements and detect the network traffic model in real time, and dynamically adjust thresholds. This achieves high throughput with zero packet loss and maximizes performance.
- Enhanced reliability
Ensuring service system reliability is the foundation of storage. For example, a switchover within seconds is one of the key assurances for reliability. However, conventional Ethernet cannot proactively detect and generate notifications of faults. When a fault occurs, link switchover takes a long time, interrupting storage services. Huawei NoF+ Solution provides the intelligent awareness feature, which helps transition from passive response to proactive notification and identify congestion and faults in advance. When a node is faulty, services are switched within sub-seconds, ensuring system stability and reliability while maintaining high performance.
- Enhanced usability
For storage, conventional Ethernet requires manual configuration node by node, which is complex and error-prone. In the Huawei NoF+ Solution, storage devices and switches intelligently collaborate, and storage devices support plug-and-play and one-click capacity expansion. Service changes can be automatically synchronized across the entire network after they are configured on a single node, improving service provisioning efficiency and enhancing usability in future construction.
Huawei's NoF+ Solution is the industry's only lossless Ethernet solution. It is based on all-flash DCs and hyper-converged Ethernet, and leverages the latest OceanStor Dorado all-flash storage system and CloudEngine DC storage network switches. The solution delivers drastically improved performance, reliability, and usability on dedicated storage networks, making it ideal for storage networks in the all-flash storage era.
Network Architecture of NoF+
- Service network: a communication network used by service servers to provide services for external systems. This network is connected to external networks.
- Computing network: a physical network formed by compute nodes running background services for the Online Transaction Processing/Online Analytical Processing (OLTP/OLAP) database. It connects to the service and storage networks using different NICs, physically isolating the two networks to avoid mutual impact.
- Storage network: a communication network used by computing servers to access storage data. This network is usually an independent physical network. To ensure high data reliability, the storage network supports DC-level Disaster Recovery (DR) and intra-city active-active deployment. If a device fault occurs in the service system or even a single DC is faulty, services are automatically switched, achieving zero Recovery Point Objective (RPO) and approximately zero Recovery Time Objective (RTO).
DCs need to communicate with each other for DR. For example, two DCs in the same city back each other up and are both running. When a device in one DC or even the entire DC is faulty, services are automatically switched to the other DC — something that was not possible for traditional DR centers. As such, users are assured rock-solid reliability, enhanced service continuity, and higher storage resource utilization.
Key Technologies of NoF+
Flow Control
End-to-end flow control is used to suppress the data transmission rate of the traffic sender so that the traffic receiver can receive all packets, thereby preventing packet loss in cases where the traffic receiving interface is congested. Huawei provides PFC deadlock detection and PFC deadlock prevention mechanisms to prevent PFC deadlocks.
- Priority-based Flow Control (PFC): is the most widely used flow control technology. When a PFC-enabled queue on a device is congested, the upstream device stops sending traffic in the queue, implementing zero packet loss.
- PFC deadlock detection: PFC deadlocks are monitored throughout the process. If a device continuously receives PFC frames within the deadlock detection period, the device can remove PFC deadlocks by not responding to the PFC frames.
- PFC deadlock prevention: This function enables devices to identify service flows that may cause a PFC deadlock, modify the queue priority, and change the PFC backpressure path to prevent loops caused by PFC frames, thereby preventing PFC deadlocks.
Congestion Control
Congestion control is a method of controlling the total amount of data entering a network to keep the network traffic at an acceptable level. The difference between congestion control and flow control is that flow control applies to the traffic receiver, whereas congestion control applies to networks. Congestion control requires the collaboration between the forwarding device, traffic sender, and traffic receiver, and uses congestion feedback mechanisms to adjust traffic on the entire network to mitigate congestion.
Mainstream congestion control technologies include:
- Explicit Congestion Notification (ECN): When congestion occurs on a network, ECN enables the traffic receiver to detect congestion on the network and notify the traffic sender of the congestion. After receiving a notification, the traffic sender reduces the packet sending rate. This prevents packet loss due to congestion and maximizes the network performance.
- AI ECN: Using the iLossless algorithm, AI ECN enables the device to perform AI training based on the traffic model on the live network, predict network traffic changes and the optimal ECN thresholds in a timely manner, and adjust the ECN thresholds in real time based on traffic changes on the live network. In this way, the lossless queue buffer is accurately managed and controlled, ensuring the optimal performance across the entire network. In addition, AI ECN can be combined with queue scheduling to implement hybrid scheduling of TCP and RoCEv2 traffic on the network. This ensures lossless transmission of RoCEv2 traffic and achieves low latency and high throughput, achieving optimal performance of lossless services.
- Network-based Proactive Congestion Control (NPCC): NPCC is a proactive congestion control technology centering on network devices. It intelligently identifies the congestion status of ports on network devices, enables devices to proactively send Congestion Notification Packets (CNPs), and accurately controls the rate of RoCEv2 packets sent by servers. This maintains a constantly proper transmission rate (both in scenarios where congestion occurs or is relieved), achieving low latency and high throughput of RoCEv2 services in long-distance transmission scenarios such as DCI.
iNOF
iNOF technology allows a device to quickly manage and control hosts, and applies intelligent lossless network technology to storage systems to implement the convergence of computing and storage networks.
A storage system that stores large amounts of data usually needs to manage many hosts, and a growing number of new hosts are connected to network devices in the system. To enable intelligent lossless network technology to better serve the storage system, iNOF technology is introduced to quickly manage and control hosts connected to the network. An iNOF-enabled device can detect a new host immediately after it connects to the network, and intelligently adjusts configurations of the intelligent lossless network. Additionally, the device can notify the storage system of host information, and help the storage system manage hosts.
iNOF can collaborate with the Storage Network Smart Discovery (SNSD) feature of OceanStor Dorado to support plug-and-play and one-click installation and link setup, which is simple and efficient. When SNSD is enabled on a RoCE port, hosts detect the status changes for all logical ports of the RoCE port and determine whether to automatically establish or tear down connections based on logical port states.
Core Components of NoF+
CloudEngine DC Storage Network Switches
Huawei's CloudEngine DC storage network switches are next-generation switches designed for DC all-flash storage networks. They feature high performance, high reliability, low delay, and easy O&M. These switches both support the innovative iLossless algorithm and enable NVMe to run more efficiently, unleashing the full potential of all-flash storage. In addition, the switches support the NoF+ technology to facilitate plug-and-play of storage networks and quick fault detection.
- CloudEngine 16800 equipped with SAN series LPUs (CEL72XS-SAN and CEL48CQ-SAN)
- CloudEngine 8850-SAN
- CloudEngine 6860-SAN
The three models work together to build an intelligent lossless DCN solution, meeting all-flash storage network requirements.
Visit CloudEngine Data Center Switches for more information.
CloudEngine DC storage network switches
OceanStor Dorado All-Flash Storage Systems
OceanStor Dorado 6800/18500/18800 V6 High-End Intelligent All-Flash Storage
OceanStor Dorado 6800, 18500, and 18800 V6 are Huawei's next-generation all-flash high-end storage systems designed for midsize and large enterprise DCs. Their design takes into account today's usage and future trends in storage technologies. These storage systems focus on core services (such as enterprise-level, virtual, and cloud DCs), aiming to provide excellent performance, robust reliability, and high efficiency.
OceanStor Dorado uses the next-generation SmartMatrix architecture, the only architecture in the industry that can ensure service continuity if one of two controller enclosures or seven of eight controllers fail. This assures reliability. In addition, OceanStor Dorado is powered by AI chips and can meet the requirements of various service applications for large databases in DCs, including OLTP, OLAP, HPC, digital media, Internet operation, centralized storage, backup, DR, and data migration.
OceanStor Dorado delivers excellent storage services for DCs along with various comprehensive data backup and DR solutions to ensure smooth and secure service operations. Also, OceanStor Dorado is easy to manage and convenient to maintain both locally and remotely, significantly reducing management and maintenance costs.
Visit OceanStor Dorado All-Flash Storage for more information.
OceanStor Dorado 6800/18500/18800 V6
OceanStor Dorado 5300/5500/5600 V6 Mid-Range Intelligent All-Flash Storage
OceanStor Dorado 5300, 5500, and 5600 V6 storage systems are next-generation all-flash storage products for mission-critical enterprise services. They are supercharged with the FlashLink intelligent algorithm designed for flash memory. For medium and large enterprises, these systems deliver large-capacity data storage, high-speed data access, high availability, high utilization, energy saving, and ease-of-use.
OceanStor Dorado offers comprehensive and superb solutions that deliver industry-leading performance thanks to a variety of mechanisms that boost efficiency. These solutions help customers maximize their Return on Investment (ROI) and meet the requirements of large database application scenarios such as OLTP, OLAP, HPC, server virtualization, and Virtual Desktop Infrastructure (VDI).
In addition to providing enterprise users with high-performance and efficient storage services, OceanStor Dorado supports advanced data backup and DR technologies, ensuring secure and smooth operation of data services. Also, OceanStor Dorado is easy to manage and convenient to maintain both locally and remotely, significantly reducing management and maintenance costs.
Visit OceanStor Dorado All-Flash Storage for more information.
OceanStor Dorado 5300/5500/5600 V6
- Author： Zhu Chunrong
- Updated on： 2023-08-07
- Views： 2931
- Average rating：
