CodePMP: A Scalable Preference Model Pre-training for Supercharging Large Language Model Reasoning

Practical AI Solutions for Enhancing Large Language Model Reasoning Challenges in Improving LLMs' Reasoning Improving the reasoning abilities of Large Language Models (LLMs) for complex logical and mathematical tasks is difficult due to the lack of high-quality preference data for fine-tuning reward models (RMs). Addressing Data Efficiency with CodePMP CodePMP is a new pretraining method that creates preference data from publicly available source code designed for reasoning tasks, making RM fine-tuning more efficient and scalable. Key Components of CodePMP CodePMP includes Reward Modeling (RM) and Language Modeling (LM) components, training models on code-preference pairs and selected responses to enhance reasoning performance. Significant Improvements in Reasoning Performance CodePMP has shown enhanced accuracy and performance in mathematical and logical reasoning tasks, offering a cost-effective solution for boosting LLM capabilities. Scalable and Efficient Approach CodePMP provides a scalable and efficient method to enhance reasoning abilities in large language models, demonstrating strong improvements across various reasoning domains. For more information and consultation: AI Lab on Telegram @itinai Twitter: @itinaicom