Liquid AI, a startup based in Boston that originated from the Massachusetts Institute of Technology (MIT), is striving to revolutionize the tech industry by moving away from the widely used Transformer architecture that powers popular large language models (LLMs) like OpenAI’s GPT series and Google’s Gemini family.
Recently, the company unveiled “Hyena Edge,” a novel convolution-based, multi-hybrid model tailored for smartphones and other edge devices ahead of the International Conference on Learning Representations (ICLR) 2025 in Vienna, Austria.
Hyena Edge stands out by surpassing strong Transformer baselines in both computational efficiency and language model quality. Real-world tests conducted on a Samsung Galaxy S24 Ultra smartphone demonstrated lower latency, reduced memory footprint, and superior benchmark results compared to a parameter-matched Transformer++ model.
Unlike conventional small models optimized for mobile deployment, such as SmolLM2, Phi models, and Llama 3.2 1B, Hyena Edge departs from attention-heavy designs. Instead, it strategically substitutes two-thirds of grouped-query attention (GQA) operators with gated convolutions from the Hyena-Y family.
This innovative architecture is a product of Liquid AI’s Synthesis of Tailored Architectures (STAR) framework, which utilizes evolutionary algorithms to automatically design model backbones. STAR explores various operator compositions to optimize for hardware-specific objectives like latency, memory usage, and quality.
To confirm Hyena Edge’s suitability for real-world use, Liquid AI conducted tests directly on the Samsung Galaxy S24 Ultra smartphone. The results exhibited up to 30% faster prefill and decode latencies compared to the Transformer++ model, with latency advantages becoming more pronounced at longer sequence lengths. Additionally, Hyena Edge consistently consumed less RAM during inference across all tested sequence lengths, making it a compelling choice for resource-constrained environments.
Hyena Edge underwent training on 100 billion tokens and was evaluated across standard benchmarks for small language models. The model either matched or exceeded the performance of the GQA-Transformer++ model on every benchmark, displaying improvements in perplexity scores on Wikitext and Lambada, as well as higher accuracy rates on PiQA, HellaSwag, and Winogrande.
Liquid AI intends to release a series of Liquid foundation models, including Hyena Edge, as open-source projects in the coming months. The company aims to develop efficient general-purpose AI systems that can scale from cloud datacenters to personal edge devices.
The success of Hyena Edge in terms of performance metrics and automated architecture design positions Liquid AI as a prominent player in the evolving AI model landscape. With the increasing demand for sophisticated AI workloads on mobile devices, models like Hyena Edge could establish a new standard for edge-optimized AI capabilities. The modern world is filled with a multitude of opportunities and challenges, all of which require individuals to adapt and evolve in order to thrive. In this fast-paced and ever-changing environment, it is essential for individuals to constantly learn new skills and stay ahead of the curve.
One key skill that is becoming increasingly important in today’s world is adaptability. The ability to adapt to new situations, challenges, and environments is crucial for success in both personal and professional life. Those who are able to quickly learn new skills, adjust to change, and think on their feet are more likely to succeed in today’s competitive landscape.
Adaptability is not just about being able to react to change, but also about being proactive and anticipating potential challenges. By developing a mindset that is open to new ideas and experiences, individuals can better navigate the complexities of the modern world.
One way to cultivate adaptability is through continuous learning. By seeking out new knowledge and skills, individuals can expand their horizons and become more versatile in their abilities. This can involve taking courses, attending workshops, reading books, or simply being open to new experiences.
Another important aspect of adaptability is resilience. In a world that is constantly changing, setbacks and failures are inevitable. Those who are able to bounce back from adversity and learn from their mistakes are more likely to succeed in the long run.
In addition to resilience, it is also important for individuals to be willing to take risks and step out of their comfort zones. By trying new things and pushing themselves beyond their limits, individuals can discover new talents and capabilities that they never knew they had.
Ultimately, adaptability is a mindset that can be cultivated and developed over time. By being open to new experiences, continuously learning, and being willing to take risks, individuals can thrive in today’s rapidly changing world. Those who are able to adapt and evolve will be best positioned to succeed in the future.
