Browsed by
Tag: 40G QSFP+

Things You Should Know About InfiniBand

Things You Should Know About InfiniBand

In today’s high speed information era, more and more people are expecting much for high bandwidth and low latency of Internet, which are the two most common parameters used to compare link performance. To cater to most people’s requirements, InfiniBand is designed to take advantage of the world’s fastest interconnect, supporting up to 56Gb/s and extremely low application latency. So what is InfiniBand exactly? You may find answer in this post.

Introduction to InfiniBand

InfiniBand (IB) is a computer-networking communication standard used in high-performance computing that features very high throughput and very low latency. It is used for data interconnect both among and within computers. InfiniBand is also utilized as either a direct, or switched interconnect between servers and storage systems, as well as an interconnect between storage systems.

Basic InfiniBand Structure

InfiniBand creates a private, protected channel directly between the nodes via switches, and facilitates data and message movement without CPU involvement with Remote Direct Memory Access (RDMA) and send/receive offloads that are managed and performed by InfiniBand adapters. The adapters are connected on one end to the CPU over a PCI Express interface and to the InfiniBand subnet through InfiniBand network ports on the other. This provides distinct advantages over other network communications protocols, including higher bandwidth, lower latency, and enhanced scalability.

Basic InfiniBand Structure

How InfiniBand Works

Instead of sending data in parallel, which is what PCI does, InfiniBand sends data in serial and can carry multiple channels of data at the same time in a multiplexing signal. The principles of InfiniBand mirror those of mainframe computer systems that are inherently channel-based systems. InfiniBand channels are created by attaching host channel adapters (HCAs) and target channel adapters (TCAs) through InfiniBand switches. HCAs are I/O engines located within a server. TCAs enable remote storage and network connectivity into the InfiniBand interconnect infrastructure, called a fabric. InfiniBand architecture is capable of supporting tens of thousands of nodes in a single subnet.

InfiniBand architecture

Features and Advantages of InfiniBand

InfiniBand has some primary advantages over other interconnect technologies.

  • Higher Bandwidth—InfiniBand constantly supports the highest end-to-end bandwidth, towards the server and the storage connection.
  • Lower latency—RDMA zero-copy networking reduces OS overhead so data can move through the network quickly.

bandwidth and latency

  • Enhanced scalability—InfiniBand can accommodate flat networks of around 40,000 nodes in a single subnet and up to 2^128 nodes (virtually an unlimited number) in a global network, based on the same switch components simply by adding additional switches.
  • Higher CPU efficiency—With data movement offloads the CPU can spend more compute cycles on its applications, which will reduce run time and increase the number of jobs per day.
  • Reduced management overhead—InfiniBand switches can run in Software-Defined Networking (SDN) mode, allowing them to run as part of the fabric without CPU management.
  • Simplicity—InfiniBand is exceedingly easy to install when building a simple fat-tree cluster, as opposed to Ethernet which requires knowledge of various advanced protocols to build an IT cluster.
Summary

InfiniBand is a high-performance, multi-purpose network architecture based on switched fabric. It has become a leading standard in high-performance computing. Over 200 of the world’s fastest 500 supercomputers use InfiniBand. If you are planning to deploy InfiniBand, feel free to ask FS.COM for help. We have 40G QSFP+ modules compliant to InfiniBand standard and various optical fiber cables for you to choose. FS.COM, as a company specialized in optical communications, offers customized network solution for each customer. You will find a best solution here.

How to Run 40GbE Over Duplex LC MMF Cabling

How to Run 40GbE Over Duplex LC MMF Cabling

The use of parallel optics in 40GbE multimode fiber cabling (MMF cabling) require more fiber strands than the 10GbE infrastructures. Thus, data center will require a cabling upgrade to meet the requirement of migration. In this case, cost is a big factor. To help user solve the fiber cost issue, many vendors developed a new transceiver alternative that allow zero-cost fiber migration by reusing the current 10Gbps multimode fiber-optic cabling plant for 40Gbps connectivity. This post will introduce three mainstream transceiver options for running 40GbE over duplex multimode fiber cable with LC connectors.

Option 1: Cisco QSFP 40G BiDi Transceiver (QSFP-40G-SR-BD)

The Cisco QSFP BiDi transceiver can transmit full-duplex 40Gbps traffic over one duplex OM3 or OM4 MMF cable with LC connector. It provides the capability to reuse 10Gbps fiber infrastructure, enabling data center operators to upgrade to 40Gbps connectivity without making any changes to the previous 10GbE infrastructures.

The working principle of Cisco QSFP BiDi transceiver is that it uses has two 20Gbps channels, each transmitted and received simultaneously over two wavelengths on a single MMF strand, as shown in the following.

QSFP BiDi

Concept of QSFP BiDi transceiver

Cisco QSFP BiDi transceiver can be supported in most Cisco switching and routing products that support 40GbE interfaces. The connection distance can reach up to 100 meters over OM3 MMF or 150 meters over OM4 MMF, which is the same as 40GBASE-SR4 QSFP+.

Note: The other version of Cisco QSFP BiDi transceiver (QSFP-40G-BD-RX) is almost the same with the QSFP-40G-SR-BD, but with link monitor hardware, such as the Cisco Nexus® Data Broker.

Buyer Guide: FS.COM offers cost-effective and highly compatible QSFP BiDi transceivers which are tested to ensure 100% Cisco compatibility.
ID# Description More Details
48722 Cisco QSFP-40G-SR-BD Compatible 40GBASE-SR Bi-Directional Duplex LC Transceiver View
65231 Cisco QSFP-40G-BD-RX Compatible 40GBASE-SR Bi-Directional Duplex LC DOM Transceiver View

 

Option 2: Juniper 40Gbps QSFP+ LX4 (JNP-QSFP-40G-LX4)

Juniper 40Gbps QSFP+ LX4 module uses the same infrastructure as 10GbE. The LX4 technology represents a new way to deploy 40GbE that meets all of the performance criteria of today’s data centers by providing 40GbE on two MMF strands with duplex LC connectors. Thus, users can simply replace existing 10GbE transceiver modules with 40GbE LX4 modules without expensive 40GbE migration cassettes and additional fiber infrastructure.

As the following picture shown, QSFP+ LX4 transceiver uses four 10Gbps channels, each transmitted and received simultaneously over four wavelengths on a single MMF strand. Similar with the 40GBASE-SR4 modules, it can also support transmission distance up to 100 meters over OM3 MMF or 150 meters over OM4 MMF cable. Juniper 40Gbps QSFP+ LX4 can be supported in many Juniper devices that support 40GbE interface, such as QFX3000 QFabric system, QFX5100 switches, and so on.

qsfp-lx4

Concept of QSFP+ LX4 transceiver

Note: The Juniper 40Gbps QSFP+ LX4 also support to run over single-mode fiber (SMF: OS1) and reach up to 2 kilometers.

Buyer Guide: FS.COM offers cost-effective and highly compatible QSFP+ LX4 transceiver which are tested to ensure 100% Juniper compatibility.
ID# Description More Details
36175 Juniper Networks JNP-QSFP-40G-LX4 Compatible 40GBASE-LX4 QSFP+ 1310nm 2km DOM Transceiver View

 

Option 3: Arista Networks QSFP-40G-UNIV Transceiver

The Arista networks QSFP-40G-UNIV is a 40GbE QSFP+ transceiver with a duplex LC connector that can be used with both MMF and SMF. Its working principle is similar with the Juniper LX4 mentioned above, having 4 channels of 10G multiplexed inside the module to transmit and receive an aggregate 40Gbps signal over 2 strands of fiber. UNIV is short for the word “universal” because of its ability to operate with both MMF and SMF without the need for any software/hardware changes to the module or any additional hardware in the network. The Arista networks QSFP-40G-UNIV can operates on OM3 or OM4 MMF for distance up to 100 meters or SMF (OS1) for distance up to 500 meters. It is noted that this QSFP module can interoperate with Cisco and other vendors standards based 40GbE LR4 optics.

Buyer Guide: FS.COM offers cost-effective and highly compatible QSFP-40G-UNIV transceiver which are tested to ensure 100% Arista Networks compatibility.
ID# Description More Details
36196 Arista Networks QSFP-40G-UNIV Compatible 40GBASE-UNIV QSFP+ 1310nm 500m DOM Transceiver View

 

For more details, please contact sales@fs.com.

40G and 100G Pluggable Optics in the Data Center

40G and 100G Pluggable Optics in the Data Center

40G and 100G are in full swing this year. All the topics, news and product solutions are around 40G and 100G, especially the pluggable optics, including transceivers, Copper DACs (Direct Attach Cables) and AOCs (Active Optical Cables). In data center optics market, 40G optics are ubiquitous and 100G optics are accelerating.

Changes of Data Center Architectures and Connections
Data center is changing both in architectures and connections. Thus, products need upgrade to meet these changes.

Architecture Changes
Data centers are becoming larger, more modular and more homogenous. Networks migrate from traditional 3-tier to flattened 2-tier topology. Workloads spread across 10s, 100s, sometimes 1000s VMs and hosts. Driven by applications like Twitter, Facebook and Youtube, higher degree of east-west traffice across networks are required.

architecture-changes

Connection Changes
Data center connections are moving from 1G and 10G, to 40G and 100G.
Within the data center rack—10G is being deployed and 25G is being standardized now. Thus, 40G and 50G may both be the next step.
Between data center racks—40G is being deployed now and 100G deployment will follow. So, the following step may be 200G or 400G.
Between data center buildings and WAN (Wide Area Network)—For long spans, 100G is being deployed and 400G is being standardized now. The following solution is expected as 800G, 1TE or 1.6TE.

40G and 100G Pluggable Optics in the Data Center

40G vs. 100G Within the Data Center
The most direct difference of 40G and 100G optics is the data rate. But actually, 40G and 100G optics have their own merits.

Characteristic of 40G Pluggable Optics

  • Lowest cost per bit
  • MMF reaches up to 300 m
  • Breaks out to 4 x 10G

Characteristic of 100G Pluggable Optics

  • Highest density per bit
  • Lowest power consumption/bit
  • MMF reaches up to 100m for now
  • Breaks out to 4 x 25GE

40G & 100G Pluggable Optics Overview
In today’s market, 40G products mainly include 40GBASE-SR4 and 40GBASE-LR4 QSFP+ modules and 40G AOCs. QSFP+ supports both 40G links between racks and high-density 10G links within the rack, especially the 40G QSFP+ breakout AOC which is an ideal solution for 40G migration.

QSFP-plus-module

100G is ready here. Tens of thousands of 100G Ethernet links deployed in core routers and carrier switches. Vast majority are CFP modules and CFP2 deployments are now starting. In addition,100G is rapidly expanding. For instance, new optical standards for the data center (100G SR4, CWDM4, PSM4) and new higher density 100G module form factors like CFP4 and QSFP28 are on the way. High port-count 100G switches are being designed and many 100G modules will be used to support high-density 10G and 25G. It is said that 100G and 4x 25G deployments are expected to grow substantially starting in 2015. 100G products mainly include 100GBASE-SR10 and 100GBASE-LR4 CFP/CFP2/CFP4 and 120G AOCs. Additionally, QSFP28 as the 100G module form factor of choice for new data center switches is also launched.

100G-CFP2

Conclusion
Large growth in web content and applications is driving such as growth in bandwidth and changes in data center traffic as well as growth in number of optical links. 40G and 100G optics development supports this growth with smaller module form factors for higher port density, lower power consumption per bit and lower cost per bit. Meanwhile, new link speeds like 25G, 50G, 200G and 400G are being standardized. The pluggable optics will continue to develop to further satisfy the data center requirement. Let’s look forward to it!

What Makes 40G AOC Is So Popular in Data Center

What Makes 40G AOC Is So Popular in Data Center

40 GbE (Gigabit Ethernet) era is already upon us to satisfy the increasing demands of higher throughputs and bandwidth. 40G AOC (Active Optical Cables), a great way to match 40 GbE, were introduced into the field of data center interconnection. As we know, 40G copper direct attach cables (Copper DAC) and 40GBASE-SR4 QSFP+ optics are also used in data center to achieve 40G interconnection. Compared with them, what makes 40G AOC so popular?

40G AOC, is a type of active optical cable for 40GbE applications that is terminated with 40GBASE-QSFP+ on one end while on the other end, in addition to QSFP+ connector, it can be terminated with SFP+ connector, LC/SC/FC/ST connector. 40G AOCs have great advantages over 40G copper DACs especially when transmission distance reaches above 7 meters. In addition to achieving longer reach, the lower weight and tighter bend radius of AOCs enable simpler cable management and the thinner cables allows better airflow for cooling.

Active_Optical_Cables

In fact, the above advantages can also be achieved by using 40GBASE-SR4 QSFP+ optics. So, why don’t we use SR4 modules but highly recommend 40G AOCs? The differences between 40G AOC and QSFP+ optics will be analyzed in the following:

40GBASE-SR4-QSFP

Cost: 40G AOCs cost lower than SR4 modules and do not need to use with extra fiber patch cables. In particular, 40G breakout AOCs, such as 40G QSFP+ to 4 x SFP+ or 40G QSFP+ to 8 x LC AOCs are cost-effective solutions to achieve 40G migration. Additionally, using AOCs, there are no cleanliness issues in optical connector and there is no need to do termination plug and test when troubleshooting, which can help user save more time and money.

Insertion Loss & Return Loss: Under the same case of transmission distance, the repeatability and Interchangeability performances of SR4 module interface are not good as 40G AOC. What’s more, when different fiber optic patch cables plug into the module, it will have the different insertion loss and return loss. Even for the same module, this issue is existed. Of course, the related metrics such as the testing eye pattern will have no significant changes so long as the variation in and conformed to the scope. In contrast, an AOC with good performance is more stable and has better swing performance than SR4 modules in this situation. The following table shows the result of the repeatability test of SR4 module. From the data, it is clear to see that the repeatability performance of SR4 module is not stable.

Test Result of 40GBASE-SR4 QSFP+ Repeatability
CH1(dBm) CH2(dBm) CH3(dBm) CH4(dBm)
1 -0.3 -0.34 -3.7 -0.03
2 -0.35 -0.33 -3.8 -0.1
3 -0.38 -0.34 -3.83 -0.14
4 -0.23 -0.37 -3.6 -0.02
5 -0.39 -0.18 -3.8 -0.13
6 -0.13 -0.37 -3.59 -0.04
Max variation 0.26 0.19 0.29 0.1

 

Four-Quadrant Test: The so called four-quadrant test is a testing under four combinations of input voltage and signal amplitude which are used to ensure the product to keep better performance even under the lowest and highest voltage and temperature situation. Four-quadrant test in wide temperature range is used to test the MTP/MPO interface and optical cable of AOC in order to ensure them not to be melted at a high temperature. Generally, the current products of AOC can all satisfy this demand. In addition, as an integration product, the performance of AOCs is more stable than SR4 modules which should be used with indeterminacy-performance MTP/MPO connectors. Unlike SR4 module, the quality index of AOCs is judged by electric eye pattern but not by light eye pattern.

Digital Diagnostic Monitoring (DDM): DDM can help end users to monitor real-time parameters of the modules. Such parameters include optical output power, optical input power, temperature, laser bias current, and transceiver supply voltage etc. 40GBASE-SR4 QSFP+ modules with DDM function can ensure it’s optimal coupling by the ADC (analog to digital converters) value of real-time monitoring receiver when receive coupling. Thus, SR4 modules have better receiving sensitivity than AOC. However, at present, both SR4 module and AOC can not reach the function of real-time power monitoring.

Transmission Distance: When transmitting over OM3 fiber, there is no significant difference between SR4 module and 40G AOC. But SR4 modules can control the performance better than AOCs. Moreover, proposals for transmission distance that is longer than 300 m will be SR4 module in order to ensure a good performance.

As was mentioned above, AOCs have better consistency and repeatability cabling performance. With the integration and sealed design, AOCs can avoid the influence of environment and vibration. Additionally, for troubleshooting, AOCs are more easier to manage. Because users do not need to do a seires of termination plug and test on-site as they do when using SR4 modules and patch cables. Thus, AOCs are highly recommended to use in data center interconnection.

Article Source: Fiberstore Blog

Comparison of 40G QSFP+ LR4 and SR4 Optics

Comparison of 40G QSFP+ LR4 and SR4 Optics

At present, for 40GbE, there are several kinds of transmission solutions. People usually use the 4 x 10 GE Breakout cable, 40G QSFP+ module or 40G CFP modules to achieve the 40GbE. Among them, the first two kinds of solutions are most widely used for nowaday 40 Gigabit Ethernet. For aggregation and core native 40 GbE, the 40G QSFP+ module is an ideal choice. There are many different variants of 40G QSFP+ modules which are defined by IEEE 802.3ba to support different applications. 40GBASE-LR4 (40G-QSFP-LR4) and 40GBASE-SR4 (40G-QSFP-SR4) QSFP+ optics are the most commonly used specifications of 40G QSFP+ optics. So, what are the differences between them on the module structure and applications?

The IEEE 802.3ba committee ratified the 40 Gigabit Ethernet standard in June, 2010. Along with the general specification, it defined a number of fiber optic interfaces. These standard interfaces attempted to satisfy a number of different objectives including support for multi-mode fiber and single-mode fiber compatibility. 40BASE-LR4 and 40BASE-SR4 QSFP+ optics both comply the IEEE 802.3ba and QSFP Multisource Agreement (MSA), but for different applications according to their features and parameters. Here are some detailed differences between 40BASE-LR4 and 40BASE-SR4 QSFP+ modules.

Application
  • 40GBASE-LR4 QSFP+ transceivers are most commonly deployed between data-center or IXP sites with single mode fiber.
  • 40GBASE-SR4 QSFP+ transceivers are used in data centers to interconnect two Ethernet switches with 12 lane ribbon OM3/OM4 cables.
Fiber Type and Connector
  • 40GBASE-LR4 QSFP+ transceivers support link lengths of up to 10 kilometers over a standard pair of G.652 single-mode fiber with duplex LC connectors.
  • 40GBASE-SR4 QSFP+ transceivers are generally enable high-bandwidth 40G optical links over 12-fiber parallel fiber terminated with MPO/MTP multifiber connectors.
QSFP+ connectors
Wavelength (nm)
  • 40GBASE-LR4 QSFP+ (Multiplexing and demultiplexing of the four wavelengths are managed within the device): 1271 nm, 1291 nm, 1311 nm, 1331 nm
  • 40GBASE-SR4 QSFP+: 4 x 850 nm
Transmission Distance
  • 40GBASE-LR4 QSFP+ (CWDM)
    – 10 km on single-mode– 4λ x 10G 1300 nm region
  • 40GBASE-SR4 QSFP+ (parallel optics)
    – 100 m on OM3–10G on 4 fibers per direction
    – 150 m on OM4 –10G on 4 fibers per direction

Obviously, the 40G QSFP LR4 is designed for long transmission used with single-mode fiber while 40G QSFP SR4 is for short distance with OM3/OM4. And “4” here means four lanes. Additionally, they are in different wavelengths and with different connectors. When they used in actual network, they are used different technology.

Ok, the article will be end here as today’s topic is simple. I hope that this post will give you some help and of course due to the limited knowledge of the author, there are some inevitable errors and omissions existing in this paper. Futhermore, if you are very interested in this topic and want to gain more information about it, I suggest you to read the IEEE 802.3ba and QSFP Multisource Agreement (MSA).


Related Recommendation

Fiberstore 40GBASE QSFP+ solution offers customers a wide variety of high-density and low-power 40 Gigabit Ethernet connectivity options for data center, high-performance computing networks, enterprise core and distribution layers, and service provider applications. Recently, Fiberstore has launched a special offer for the 40GBASE-LR4 and 40GBASE-SR4 QSFP+ transceiver modules. There is a large price cut on both of them which can definitely save your budget up to 30% off for the fiber project. In addition, there are also a promotion for MTP patch cables. For more information, please visit the latest news of Fiberstore.