Browsed by
Author: Serenity Huang

400G CFP8 FR8 and LR8 Transceivers with PAM4 Technology

400G CFP8 FR8 and LR8 Transceivers with PAM4 Technology

400G Ethernet technology is demonstrated at the Optical Networking and Communication Conference & Exhibition (OFC) 2017, and new module types like 400G CFP8 FR8 and LR8 with pulse amplitude modulation (PAM4) technology are announced. Why 400G now? While we’re still considering adopting 100G in our business, some hyperscale data centers are in urgent demand of improvements in speed and density. So component makers worked out the 400G Ethernet solutions for this group, of which CFP8 FR8 and LR8 are two optical transceivers using 50G PAM4 technology.

How Does 400GBASE-FR8/LR8 CFP8 Work?

Before going to details about the CFP8 FR8 and LR8, let’s first have a look at the form factor CFP8 for 400G Ethernet. CFP8 is the 400G form factor type specified by the CFP multi-source agreements (MSA). It has the same size as CFP2 module but supports 4x100G and 400G, which is 4 times the CFP2’s 1x100G. The CFP8 uses 16x25G electrical I/O connector. Optical specifications of CFP8 are following other standards organizations like IEEE and ITU-T. 400GBASE-FR8/LR8 is one of the IEEE specifications supported by CFP8.

400G form factor CFP8 FR8 LR8 dimension

Figure 1: 400G form factor CFP8 FR8/LR8 dimension.

CFP8 FR8/LR8 operates over single-mode fiber and has LC duplex connector. FR8 supports 2km reaches and LR8 supports 10km reaches. As the following diagram shows, FR8/LR8 uses 8x50G PAM4 WDM technology. The output signal is multiplexed into a single fiber, allowing it to transmit at 400Gb/s, and the receiver de-multiplexes the input signal to 8 optical channels of 50Gb/s. The 400G CFP8 FR8/LR8 consumes less than half the power per Gigabit compared to a 100G CFP4 MSA.

400GBASE FR8 LR8 CFP8 block diagram

Figure 2: 400G CFP8 FR8/LR8 block diagram.

Enabling Technology of 400G CFP8 FR8 and LR8 Transceivers

The MSA-compliant 400G CFP8 FR8 and LR8 is enabled by 50G PAM4 technology. PAM4 is a modulation format that has been adopted by IEEE for 50Gb/s per lane Ethernet signaling. The PAM4 technology used for CFP8 is very different from the binay non-return to zero (NRZ) which is the usually used mechanism in serial communications. By driving the laser to generate one of the two output amplitudes, NRZ has two signaling levels. PAM4 has four distinct levels to encode two bits of data, essentially doubling the bandwidth of a connection, i.e., when used for same bandwidth optical devices, PAM4 doubles the transmission capacity.

2-level NRZ and 4-level PAM4

Figure 3: 2-level NRZ and 4-level PAM4.

In addition to 400GE, PAM4 is also expected to become a good fit for other high-speed serial interfaces like 50GE, 100GE, 200GE and Fibre Channel. Also a more optimized technology 100G PAM4 is in development, which will also be used for 400G Ethernet with higher performance.

Summary

For 400G Ethernet, PAM4 is the most feasible solution considering the its simplicity and lower power consumption, and it is currently the most accessible approach compared with other technologies. The CFP8 FR8/LR8 has been demonstrated at the OFC 2017, and the leading companies are working on bringing it to market while keeping the cost acceptable. It is worth to believe that with their efforts the price of 100G devices will drop greatly and 400G will be deployed not only in hyperscale data centers but also in other enterprise data centers.

RJ45 vs SFP: Which Should I Use to Connect Two Switches?

RJ45 vs SFP: Which Should I Use to Connect Two Switches?

When switches have both SFP port and Ethernet port for 1000Mbps connections, it is always the case for us to decide whether to use SFP fiber optic module or RJ45 Ethernet cable solution. Although it really depends on many factors, there are some principles that we could follow in our situation so as to meet our requirements.

 SFP vs RJ45

Common Principles for Using RJ45 vs SFP

The first principle is appropriate for any cases of 1000Mbps deployment. That is when the distance of the run is over 328 ft/100 m, fiber SFP module must be considered instead of copper RJ45 port or copper SFP RJ45 module. Since 1000Mbps could only go as far as 100 m over copper cabling.

When the link distance is under 100 m, there are some other principles that we should follow when we decided to use RJ45 or SFP.

1. Considering security issue, using fiber SFP module is better than Cat5e/6 RJ45 Ethernet cable or SFP RJ45 module in some of the worst circumstances. RJ45 is using electric current so it is easier to caught on a fire/lightning while fiber SFP module has no such problem. Also using fiber optic module is safer in terms of Internet security, because intercepting data transmissions by tapping copper cable is less complicated than tapping on fiber optic cable. So when security could be a problem in the connection, using fiber SFP is better than RJ45 port.

2. Reliability is also a key factor in transmission. Fiber is often considered more reliable due to its properties. Copper RJ45 Ethernet cable has the risk of interference since it uses electric signal. The issue is usually called EMI (Electromagnetic Interference) when the electric signal is affected by some external elements, such as something with high voltage. So when the cable run is going through some places that the electric signal can be interfered, choose fiber rather than copper RJ45 cabling.

copper cables use electric signal

3. Future consideration. If you’re planning to move to higher bandwidth, e.g. 10Gbps, in the near future, fiber optic cable is more future-proof compared with Cat5e/Cat6 RJ45 Ethernet cable.

4. Cost is also important in deployment. When the three issues above do not matter in our situation, cost is what we care. Since Rj45 Ethernet port and the SFP port actually run at the same speed, choosing the RJ45 ports to connect the switches might be more economical since Cat5e/6 cable is lower-priced than SFP transceiver module.

RJ45 vs SFP: Go to the Real Cases for 1000Mbps Switch Connections

These rules are found on the basis of some typical cases. If it is right the situation we’re in, we might feel lucky to get the direct answer. But most of the time even if we have known the principles for 1000Mbps deployment, it will still be confusing in real cases. Let’s see some cases of our fellows.

RJ45 vs SFP: Latency Makes a Difference

Case description: Right now my switches are connected on Gigabit Ethernet ports over about 300 ft or so. Would it be better to replace that line with a fiber line even though my port is only SFP and not SFP+ and locked at 1Gb? One switch houses 24 heavy active users and the other houses about 80 active devices and is my core switch. My goal is to clear any/all latency that I can between the two buildings.

1000Mbps switches are in different buildings

Case analysis: The two switches in this case are in two different buildings. The link distance is around the maximum transmission distance (100 m) for Cat5e/Cat6 Ethernet cable but does not exceed it, so that’s why the transmission was going well with the Rj45 Ethernet ports. The key purpose is about minimizing the latency. But in fact, the latency difference for using fiber will not be significant. Other factors are what should be considered since the switches are in two different buildings. Firstly, if the two buildings have separate electrical power inputs, then running copper between them could cause ground loop issues, so go fiber. Secondly, lightning strike might matter when using copper cable across the buildings, so go fiber. If neither of fiber or copper run is feasible, go wireless.

RJ45 vs SFP: Power Consumption Is a Challenge

Case description: I’m very confused. When connecting switches in the same rack it seems SFP is always used instead of a normal RJ45 Ethernet cable, even if the SFP port is only GB like the other ports on the switch. Why use SFP in that situation? SFP transceiver costs a lot.

Cisco 3560 switch RJ45 connections

Case analysis: This is about connecting 1000Mbps switches on the same rack. Fiber SFP port is used instead of RJ45 port maybe because there is electrical interference. Another reason could be the large amount of links. When there are more than 100 links, choosing SFP is more power saving than Ethernet ports. Also if there is a need to save the Ethernet ports for end-point connections, choose fiber for the switch connection. Keep in mind that we can use one type or the other but not both at the same time in a switch trunking.

10G Stacking Over Cat6a or DAC

10G Stacking Over Cat6a or DAC

We have got benefits from the stacking of 10G SFP+ switches for the past decades, and the 10G SFP+ direct attach cable (DAC) makes the stacking low-cost. But when 10GBASE-T becomes a new port choice on 10G switches, stacking on 10G switches has to meet the issue about whether to choose Cat6a or SFP+ DAC. At first people think that Ethernet cable is cheaper and easier to deploy than SFP+ DAC, but later they realized the issue is not only about the cable cost.

  10G stacking-cat6a or DAC

Comparison of Stacking 10G Over Cat6a and SFP+ DAC

Before going on, it should be noted that this discussion is focused on 10G stackable switches that have both 10GBASE-T ports and SFP+ ports (such as Cisco 550X series switches, Dell 8024).

In terms of the cost, one Cat6a cable costs just one third of a 10G SFP+ DAC of the same length. But cable price is not the only factor that contributes to the total budget in building a network. Let’s see some properties of these two technologies.

Power Consumption
SFP+ DAC Cat6a
Power Consumption Per Port 1-1.5 W 4-6 W
Max. Reach 10 m 100 m

Obviously, SFP+ DAC uses much less power than Cat6a cable. It is important to know that for every watt of power consumed, typically two additional watts are needed for cooling. If there is a lot of ports in the stacking, this power consumption will be significant. So when power consumption matters a lot in your stacking, it will be a huge saving to choose SFP+ DAC instead of Cat6a.

10G cable power consumption
More ports mean more power consumption.
Latency

Also in a case when low latency is strictly required, like in automated financial trading, it is also better to use SFP+ DAC for 10G stacking. Because for the same links and lengths, Cat6a presents higher latency than SFP+ DAC.

automated financial trading low latency
Automated financial trading requires low latency.
Distance

When you have many stacked 10G switches located at different places, i.e. at separate floors or at opposite ends of a hallway, Cat6a can achieve the stacking connection if it is less than 100 m. But typically you’ll be suggested to use fiber link, because the power consumption in long-term could exceed the extra cost for buying 10G fiber optics and cables.

10G stacking distance
The stacking distances affect the cable choice.

As for distances more than 100 m, neither SFP+ DAC nor Cat6a can work. You have to turn to fiber optic patch cable and 10G SFP+ transceiver module.

Conclusion

Considering the total cost and performance of Cat6a and SFP+ DAC for 10G stacking, SFP+ DAC appears to be a better choice in short range. The low power consumption and low latency properties can help accomplish a good stacking, and thus shortening the downtime, improving resiliency, and simplifying administration.