As the telecommunications industry continues to evolve, the focus is shifting towards the integration of AI technology into network operations. From facilitating high-capacity data transfer between hyperscale and cloud data centers to enabling AI inferencing at the network edge, the role of the network core is pivotal in reshaping business functions across various industries. This transformation will allow companies to leverage pre-trained AI models for real-time processing closer to end users, driving the need for optimized network infrastructure to support this emerging use case.
By Mattias Fridström is the Vice President and Chief Evangelist at Arelion
Many discussions within the telecommunications sector are currently centered around hyperscale and cloud data centers that handle AI training workloads, with a significant emphasis on the network core’s role in facilitating seamless data transfer. The transition towards AI inferencing at the network edge is on the horizon, promising to revolutionize business operations by enabling the utilization of pre-trained AI models for processing requests in closer proximity to end users. While inferencing demands less bandwidth compared to AI training, it is poised to drive Internet carriers towards optimizing their infrastructure for reduced latency and enhanced capacity to support this evolving paradigm.
Analysts predict that nearly half of the data center market’s $1 trillion CAPEX by 2029 will be allocated to accelerated servers optimized for AI. This forecast underscores the importance of Internet carriers adapting their architectures to meet the networking requirements essential for enterprises and hyperscalers to maximize their AI investments. The dynamic and latency-sensitive nature of AI workloads presents challenges to traditional networks, necessitating innovative solutions to mitigate bottlenecks and ensure optimal performance. How will the evolution of inferencing impact network infrastructure and address issues such as latency, jitter, and other risks?
Similar to Content Delivery Networks (CDNs), AI inferencing demands fast and localized delivery, albeit with a higher degree of dynamism and less cacheability due to its context-specific nature. Reliable network performance is paramount for the seamless operation of real-time AI inferencing, highlighting the need for telecom operators to optimize key networking aspects such as reach, capacity, scalability, and more to meet the decentralized demands of AI inferencing.
Backbone networks will play a crucial role in disseminating inferencing responses to end users through strategically positioned Points-of-Presence (PoPs) that ensure optimized connectivity in both established and emerging markets. The success of inferencing operations hinges on an expansive network reach that enables carriers to localize AI workloads and provide low-latency delivery to end users by leveraging the vast network infrastructure of the global Internet.
Ensuring reliability in network operations is essential for delivering model outputs to the edge, allowing companies to leverage high-availability services effectively. By implementing network diversity and latency-based segment routing, Internet carriers can enhance reliability and seamlessly reroute AI traffic through alternate low-latency paths in the event of disruptions, safeguarding real-time AI operations from potential threats like geopolitical interference, weather-related incidents, or accidental fiber cuts.
Enhancing Scalable Capacity through Optical Innovations
In tandem with data center advancements to accommodate emerging applications, Internet carriers are revolutionizing their optical networking infrastructure to meet the demands of AI use cases for scalable capacity. The integration of 400G coherent pluggable optics in backbone networks through open optical line systems enables carriers to address the evolving capacity and scalability requirements of their customers. Unlike traditional transponders, coherent pluggables offer a modular and software-driven approach that aligns with the dynamic nature of AI workloads, providing real-time capacity adjustments to meet fluctuating data needs.
While inferencing operations occur at the edge, the transmission of training data back to core and cloud networks for aggregation and analysis remains essential. The adoption of 400G coherent pluggables (with 800G pluggables on the horizon) facilitates core-edge synergy by establishing high-capacity links between core, cloud, and edge nodes, enabling carriers to support AI’s varying data requirements efficiently. Additionally, these pluggables offer energy-efficient solutions that reduce space and power consumption compared to traditional transponders, contributing to the cost-effectiveness and sustainability of carrier networks amidst the energy-intensive demands of AI operations.
Unwavering Importance of Backbone Connectivity
While current AI workloads predominantly concentrate in hyperscale and cloud data centers, the transition towards inferencing signifies the next phase of AI evolution. The critical role of backbone connectivity in facilitating AI data transfer between data centers is well-established, yet its significance extends to supporting future AI functions at the network edge. By prioritizing key networking attributes, Internet carriers can establish a robust foundation for AI inferencing, empowering hyperscalers, cloud data center operators, and enterprises to leverage scalable and reliable connectivity for unlocking the full business potential of AI technologies.
About the Author
Mattias Fridström serves as the Vice President and Chief Evangelist at Arelion, bringing over two decades of experience in the telecommunications industry. With a background in various senior roles within Telia Carrier (now Arelion), Mattias has been instrumental in driving technological innovation and advocating for the transformative potential of AI in networking.
Related
Article Topics
AI inferencing | AI networking | connectivity | data center | digital infrastructure | edge computing | edge networking | network edge
