VisOnlyQA: A New Dataset for Evaluating the Visual Perception of LVLMs (Large Vision Language Models)

**Understanding Visual Perception in LVLMs** **Recent Progress** Large Vision Language Models (LVLMs) are improving in handling tasks that combine images and text. However, they still struggle with accurately interpreting visuals, which impacts their performance in complex image-related tasks. **Current Evaluation Issues** Current datasets, like MMMU and MathVista, do not effectively test visual perception; instead, they focus on advanced reasoning. This makes it hard to judge how well LVLMs understand visual information. While basic visual tasks like counting or depth estimation are covered, these datasets lack detailed questions for proper assessment. **Introducing VisOnlyQA** To fill this gap, researchers at Penn State University developed the VisOnlyQA dataset. This dataset tests LVLMs on geometric and numerical queries within scientific figures, focusing on fine visual details with synthetic figures for better variety. Questions are easy to understand, either created manually or generated automatically. **Dataset Quality and Structure** VisOnlyQA is divided into three parts: Eval-Real, Eval-Synthetic, and Train. Each section has balanced labels and high-quality annotations, with human accuracy rates between 93.5% and 95%. **Model Performance Evaluation** The study evaluated 20 different LVLMs using VisOnlyQA, focusing on geometry, chemistry, and chart analysis. Results showed these models underperformed compared to humans, with average accuracies of only 54.2% for real data and 42.4% for synthetic data, both significantly lower than human performance. **Challenges Ahead and Future Prospects** Despite progress in model sizes, LVLMs struggle with visual perception tasks, indicating room for improvement. Current methods, like chain-of-thought reasoning, do not always enhance visual task performance, underlining the need for better training data and improved model designs. **Final Thoughts** VisOnlyQA is an essential tool for evaluating the visual understanding of LVLMs and points out where improvements are necessary. This dataset paves the way for further research and applications in the AI field. **Boost Your Business with AI Solutions** Stay competitive by using AI tools like VisOnlyQA. Here’s how to utilize AI effectively: 1. **Discover Automation Opportunities** Identify areas where AI can enhance customer interactions. 2. **Define KPIs** Ensure that your AI projects deliver measurable results. 3. **Select the Right AI Solution** Choose tools that fit your specific requirements and allow customization. 4. **Implement Gradually** Start with a pilot project, gather data, and expand your AI initiatives wisely. **Stay Connected** For expert advice on AI management, reach out at hello@itinai.com. For regular updates on leveraging AI, follow us on Telegram or Twitter @itinaicom. Explore how AI can transform your sales and customer engagement at itinai.com.