25Gbps vs. 40Gbps Ethernet

Before Ethernet won the LAN wars between Ethernet and token ring, there was a legitimate debate of speed over architecture. In the 90’s, Ethernet moved from 10 Mbps to 100 Mbps while token ring moved from 8 Mbps to 16 Mbps. While a significant difference in speed, an argument of dedicated bandwidth in token ring vs. contention-based Ethernet was interesting. The industry settled on raw speed. There’s similar debate in today’s Ethernet. Customer needing speeds above 10 Gbps must make a decision on 25 Gbps vs. 40 Gbps.

The 40 Gbps and 100 Gbps network standard have existed for several years. Both standards were ratified in 2010 after development began in 2008. Cisco released a whitepaper highlight their rationale of not waiting until 100 Gbps is ready. Chances are if you have the use case for 10 Gbps, you benefit from 40 Gbps. The question is if the proper stepping stone is 10 Gbps-to-25 Gbps vs. 10 Gbps-to-40 Gbps.

Webscale influence & optics

A group of webscale companies that included Microsoft and Google formed a consortium to promote the 25 Gbps standard. The drive toward 100 Gbps provided a design challenge to the webscale companies.

100 Gbps Ethernet is four 25 Gbps waves. 40 Gbps is four 10 Gbps lanes. In the purpose-built data centers of the webscale companies, it makes more sense to replace existing 10 Gbps interfaces with 25 Gbps interfaces. In an earlier post, I shared how FB has begun developing a switch architecture based on 25 Gbps standard and open sourcing the design. Most recently FB created a 100 Gbps switch and released the design to open source.

Since 25 Gbps and 100 Gbps switches share the same components, the two speeds share engineering overhead. Also, since 100 Gbps is four lanes of 25 Gbps, designing for aggregation is simple. In a simplified example, four top of rack (ToR) 25-Gbps switches can aggregate into a single 100 Gbps switch. Or customers can oversubscribe a single aggregation point using a divisible factor of 25 Gbps to each 100 Gbps. While not a major deciding factor, aggregation is an interesting angle.

The market

The industry has ramped up for 40 Gbps. The engineering cycles for large networking companies is 18 months. We see companies begin to react to the influence of 25 Gbps. Until recently, Cisco focused on selling what they had – 40Gbps. There was no mention of 25 Gbps as an option in the prior mentioned whitepaper.

Cisco Nexus line has begun to adopt 25 Gbps for ToR switches. While I don’t follow this market closely, I haven’t seen much in at the aggregation layer; which makes sense. 100 Gbps is the logical aggregation bandwidth for 25 Gbps ToR switches.

Network vendors struggle with the speed of innovation driven by the webscale companies. If you are in an high-bandwidth environment, you are wise to look at whitebox solutions that decouple your organization from the upgrade cycles of traditional enterprise network vendors.