AutoDroid-V2: Leveraging Small Language Models for Automated Mobile GUI Control

Revolutionizing Mobile Device Control with AutoDroid-V2 **Understanding the Challenge** Controlling mobile devices with natural language has become easier thanks to advanced AI models. However, traditional methods can raise privacy concerns and be costly, making them hard to use widely. **Previous Automation Attempts** Older tools like Siri and Google Assistant used fixed templates, which limited their flexibility. More advanced tools struggled to keep up with the constantly changing mobile apps, making automation challenging. **Introducing AutoDroid-V2** Researchers at Tsinghua University have created AutoDroid-V2, a new method that uses the coding strengths of Small Language Models (SLMs). Unlike traditional methods, AutoDroid-V2 can create and run multi-step scripts based on what users ask. This brings two main benefits: - **Efficiency**: It creates one script for multiple actions, reducing the need for constant queries and saving resources. - **Capability**: It uses the proven coding abilities of SLMs. **How AutoDroid-V2 Works** AutoDroid-V2 works in two steps: 1. **Offline Processing**: It looks at app usage history to create a detailed document for each app, which helps in generating scripts. 2. **Online Processing**: When a user asks for a task, a local LLM creates a multi-step script that is executed by a specialized interpreter. **Performance Improvements** AutoDroid-V2 has been tested on 226 tasks across 23 mobile apps with impressive results: - **Higher Task Completion**: Increases task completion rates by 10.5% to 51.7%. - **Reduced Resource Use**: Lowers input and output token use significantly and reduces processing time. - **Consistent Success Rates**: Maintains a success rate of 44.6% to 54.4% across different models. **Conclusion** AutoDroid-V2 marks a major advancement in mobile task automation with its innovative script-based approach. It improves efficiency and performance while ensuring privacy and security. Future improvements may include better handling of apps that lack structured text. **Get Involved** Stay updated by following us on Twitter, joining our Telegram Channel, and connecting with our LinkedIn Group. **Join Our Webinar** Learn how to improve AI performance while protecting data privacy. **Elevate Your Business with AI** - **Identify Automation Opportunities**: Find areas where AI can help. - **Define KPIs**: Measure AI's impact on your business. - **Select the Right AI Solution**: Choose tools that meet your needs. - **Implement Gradually**: Start small, learn, and grow. For AI management advice, contact us at hello@itinai.com. Stay connected for ongoing AI insights on our Telegram and Twitter. **Transform Your Sales and Customer Engagement** Discover how AI can improve your business processes at itinai.com.