Researchers from China Introduced a Novel Compression Paradigm called Retrieval-based Knowledge Transfer (RetriKT): Revolutionizing the Deployment of Large-Scale Pre-Trained Language Models in Real-World Applications

Researchers from China Introduced a Novel Compression Paradigm called Retrieval-based Knowledge Transfer (RetriKT): Revolutionizing the Deployment of Large-Scale Pre-Trained Language Models in Real-World Applications AI News, AI, AI tools, Aneesh Tickoo, Innovation, itinai.com, LLM, MarkTechPost, t.me/itinai 🚀 **Researchers from China Introduced a Novel Compression Paradigm called Retrieval-based Knowledge Transfer (RetriKT): Revolutionizing the Deployment of Large-Scale Pre-Trained Language Models in Real-World Applications** 🚀 Exciting news from researchers at Peking University, Meituan, Meta AI, National Key Laboratory of General Artificial Intelligence, BIGAI, and Renmin University of China! They have introduced a groundbreaking compression paradigm called RetriKT, which aims to efficiently and precisely transfer information from Large Language Models (LLMs) to small-scale models. Here's how it works: The researchers extract knowledge from the LLM to create a knowledge store. Then, the small-scale model retrieves relevant information from this store to complete its tasks. Through comprehensive tests on difficult and low-resource tasks, the researchers found that RetriKT significantly improves the performance of small-scale models compared to previous knowledge distillation techniques. **Key Contributions:** - RetriKT is a novel compression paradigm that transmits information from LLMs to small-scale models. - The researchers carefully constructed the incentive function and proposed the reinforcement learning algorithm PPO to improve the generation quality. - Through comprehensive tests, they improved the accuracy and diversity of knowledge collected from LLMs used for knowledge transfer, resulting in improved performance of small-scale models. 🔧 **Practical AI Solution:** If you're a middle manager looking to leverage AI and stay competitive, consider using the AI Sales Bot from itinai.com/aisalesbot. This solution automates customer engagement 24/7 and manages interactions across all customer journey stages. Discover how AI can redefine your sales processes and customer engagement by exploring the solutions at itinai.com. For more information and insights into leveraging AI, you can connect with us at hello@itinai.com or stay updated on our Telegram channel t.me/itinainews or Twitter @itinaicom. 🔗 **List of Useful Links:** - AI Lab in Telegram @aiscrumbot – free consultation - Researchers from China Introduced a Novel Compression Paradigm called Retrieval-based Knowledge Transfer (RetriKT): Revolutionizing the Deployment of Large-Scale Pre-Trained Language Models in Real-World Applications - MarkTechPost - Twitter – @itinaicom