2023 云栖大会番外篇：从灵骏与 SCC 之争看下一代互联，EtherNET 还是 EtherNOT

OCP

Ultra Ethernet

Best Effort vs Predictable

• Jupiter Evolving: https://www.youtube.com/watch?v=Hfl-i56hZUg&ab_channel=ACMSIGCOMM
• Traffic patterns in our DC are predictable
• Clos is great when traffic pattern is unknown
• Jupiter block-level traffic is predictable
• 什么是可预期网络？ https://www.zhihu.com/question/559661458/answer/2746425425
• GPU 大模型网络设计实现 飞书文档

光交换

• 光互连
• 光交换
• 光模块
  • 搞了半辈子网络，有些光模块，你可能没见过！ https://mp.weixin.qq.com/s/4030Q5wKvlzsLR9UMPNYeg?
  • 关于光模块，看这一篇就够啦！ https://zhuanlan.zhihu.com/p/101607360
  • What Is an Optical Module and Its FAQs https://support.huawei.com/enterprise/en/doc/EDOC1100130737
  • https://cshihong.github.io/2020/08/16/%E5%85%89%E6%A8%A1%E5%9D%97%E5%AD%A6%E4%B9%A0%E6%80%BB%E7%BB%93/
  • http://www.iccsz.com/site/cn/News/2020/04/20/20200420041614627821.htm
• OXC https://zhuanlan.zhihu.com/p/629283582
• https://cloud.google.com/blog/topics/systems/the-evolution-of-googles-jupiter-data-center-network
• 800 G 数据中心网络标准刚刚被确定 https://www.synopsys.com/zh-cn/designware-ip/technical-bulletin/benefits-800g-ethernet-implementation.html
• 交换机流量收敛 https://support.huawei.com/enterprise/zh/doc/EDOC1100023543?section=j00m
• https://en.wikipedia.org/wiki/Optical_module

Serdes

• https://en.wikipedia.org/wiki/SerDes

Architecture

• Fat Tree https://blog.csdn.net/u012925450/article/details/108493968
• DragonFly
• Clos

Hot-Interconnect

• https://zhuanlan.zhihu.com/p/659017509

Summit

• https://ultraethernet.org/

https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/theoretical_memory_usage.py#L141

https://github.com/NVIDIA/Megatron-LM/pull/617/files

实践

• 腾讯星脉网络 https://www.infoq.cn/article/ievumz2rfolepfph7mx2
• 百度网络 https://developer.baidu.com/article/detail.html?id=298969
• PXN https://developer.nvidia.com/blog/doubling-all2all-performance-with-nvidia-collective-communication-library-2-12/
• 阿里 HPN 7.0
• Rail-Optimized 互联收益分析飞书

参考资料

• 灵骏可预期网络：Built for AI Infrastructure， https://mp.weixin.qq.com/s/-QojHhmgM0-jqTYQUpkSgw
• SCC 宣传文章 https://mp.weixin.qq.com/s/zyQdGjNoUpxnlhqgzYDePQ
• OCP 2023 https://www.opencompute.org/events/past-events/2023-ocp-global-summit
• 会议回顾 https://mp.weixin.qq.com/s/kDzJ1MEaizd0fnmhuCm9PQ

bojie 的文章

• https://zhuanlan.zhihu.com/p/657498193