Can Smaller AI Models Outperform Giants? This AI Paper from Google DeepMind Unveils the Power of ‘Smaller, Weaker, Yet Better’ Training for LLM Reasoners

Practical Solutions for Training Large Language Models (LLMs) Enhancing Model Performance with Compute-Efficient Synthetic Data Training large language models (LLMs) for reasoning tasks can be challenging due to the need for compute-efficient methods to generate synthetic data that improves model performance. Traditionally, using stronger and more expensive language models (SE models) to create high-quality synthetic data for fine-tuning has been the norm. However, this approach is resource-intensive and limits the amount of data that can be generated within a fixed computing budget. Current methods for improving LLM reasoning capabilities, such as knowledge distillation and self-improvement, have proven effective but come with drawbacks, including high computational costs that restrict the volume and diversity of data produced. Researchers from Google DeepMind propose a novel approach that challenges the reliance on SE models for synthetic data generation. They advocate for using weaker but cheaper models (WC models) that, despite their lower quality, are more cost-effective and enable the generation of larger data volumes within the same computing budget. The technical details involve a comparative analysis between SE and WC models under a fixed compute budget. Experiments using the Gemma2 family of models on datasets like MATH and GSM-8K showed that WC models, represented by Gemma2-9B, outperformed SE models, represented by Gemma2-27B, in generating data that led to significant improvements in LLM performance. Using WC models for synthetic data generation proves to be more compute-efficient than relying on SE models. By generating more diverse and comprehensive training data within a fixed compute budget, WC models enable the training of stronger LLM reasoners. Discover how AI can redefine your way of work. Identify Automation Opportunities, Define KPIs, Select an AI Solution, Implement Gradually. For AI KPI management advice, connect with us at hello@itinai.com. Discover how AI can redefine your sales processes and customer engagement. Explore solutions at itinai.com. List of Useful Links: AI Lab in Telegram @itinai – free consultation Twitter – @itinaicom