What is InfiniBand?
InfiniBand is a network communication protocol that provides a switch based point-to-point bidirectional serial link structure (Fabric) between processor nodes and between processor nodes and input/output nodes (such as disks or storage). Each link has a device connected to each end of the link, so the characteristics of controlling transmission (sending and receiving) at each end are well defined and controlled.
InfiniBand creates a dedicated protected channel directly between nodes through a switch, and manages and executes remote direct memory access (RDMA) and send/receive offloading through InfiniBand adapters, facilitating the movement of data and messages. One end of the adapter is connected to the CPU through a PCI Express (PCIe) interface, and the other end is connected to the InfiniBand subnet through an InfiniBand network port. Compared to other network communication protocols, this provides significant advantages, including higher bandwidth, lower latency, and enhanced scalability.
The main advantages of InfiniBand include:
• Higher Throughput – InfiniBand continues to support the highest end-to-end throughput. For server and storage connectivity, InfiniBand introduced 40Gb/s (QDR) to the market in 2008, while Ethernet only supported 10Gb in 2011 and Fibre Channel only supported 8Gb. Introducing 56Gb/s (FDR) to the market, while Ethernet supports 40Gb and Fibre Channel only supports 16Gb/100Gb/s (EDR) InfiniBand products were launched in 2014, and 200Gb/s (HDR) will be launched in the coming years, Utilize competitive structures to maintain market disparities.
• Low latency – RDMA zero copy network reduces operating system overhead, allowing data to move quickly across the network.
• Enhanced scalability – InfiniBand can accommodate a flat network of approximately 40000 nodes in a single subnet, and can accommodate up to 2 ^ 128 nodes (actually an unlimited number) in a global network, with just the need to add additional switches.
• Higher CPU efficiency – By moving and offloading data, CPUs can spend more computing cycles on their applications, which will reduce runtime and increase the number of jobs per day.
• Reduced management overhead – InfiniBand switches can run in software defined network (SDN) mode, allowing them to run as part of the fabric without the need for CPU management.
• Simplicity – When building a simple fat tree cluster, InfiniBand is very easy to install, while Ethernet requires knowledge of various advanced protocols to build an IT cluster.
Most importantly, InfiniBand provides better investment returns, providing higher throughput and CPU efficiency at competitive prices, and achieving higher productivity at lower cost per endpoint.
