Summary:
1. Red Hat introduces the Red Hat AI Inference Server, enhancing generative AI applications’ speed and efficiency.
2. The software compresses AI models for better performance and optimizes processor memory for faster inferencing.
3. Red Hat also announces advancements in virtualization market growth and key announcements at the Red Hat Summit 2025.
Article:
Red Hat recently unveiled the Red Hat AI Inference Server at the Red Hat Summit in Boston, a groundbreaking software that boosts the speed and efficiency of generative AI applications. By utilizing technology from the vLLM project and Neural Magic, this new AI inference server enables enterprises to run AI models faster and more effectively by compressing trained models and optimizing processor memory. This advancement signifies Red Hat’s commitment to helping businesses enhance their AI investments by delivering maximum performance through software optimization.
Moreover, Red Hat’s AI Inference Server excels in optimizing the use of GPUs through innovative techniques like improved memory management and continuous batching. The software supports various AI accelerators such as AMD, Nvidia GPUs, Intel’s Gaudi AI accelerators, and Google TPUs, ensuring versatile and efficient AI model optimization. Pre-optimized models running on vLLM have shown significant efficiency improvements, delivering two to four times more token production, as highlighted by Brian Stevens, Red Hat’s senior vice president and AI chief technology officer.
Furthermore, Red Hat’s efforts in the virtualization market have garnered significant growth, with over 150% increase in Red Hat OpenShift Virtualization deployments since 2024. To cater to a wider audience, key cloud providers like Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure are making Red Hat OpenShift Virtualization available through technology or public previews. Red Hat’s virtualization software is now also available on Amazon Web Services (AWS) and IBM Cloud, showcasing the company’s commitment to meeting the diverse needs of customers in the virtualization space.
At the Red Hat Summit 2025, the company made several key announcements, including the launch of Red Hat Enterprise Linux 10 with enhanced security features and container image deployment capability. Red Hat also introduced the llm-d open source community to scale inferencing and Lightspeed generative AI assistants for Enterprise Linux 10 and OpenShift environments. Additionally, the unveiling of the Red Hat Advanced Developer Suite and more cloud-related news further solidifies Red Hat’s position as an industry leader in AI and virtualization technologies.
In conclusion, Red Hat’s continuous innovation in AI inference, virtualization, and key technology announcements reaffirms their dedication to empowering enterprises with cutting-edge solutions for enhanced performance and efficiency in the digital era.
