ScaleCUA introduces a large-scale dataset and framework to advance the capabilities of computer use agents (CUAs). Developing robust CUAs is challenging due to the scarcity of computer use data. ScaleCUA addresses this by providing a dataset spanning six operating systems and three task domains, enabling the training of general-purpose agents that can operate seamlessly across platforms. This work demonstrates the power of data-driven scaling for creating versatile CUAs.
ScaleCUA offers a comprehensive dataset covering diverse operating systems and tasks, facilitating the training of cross-platform agents. The project includes a versatile training framework supporting multiple VLMs like Qwen2.5-VL and InternVL. It features an interactive playground for hands-on experimentation and an online evaluation suite for standardized performance assessment. The project fosters rapid development through a modular structure with evaluation, playground, and training components.
- Cross-Platform Support: Operates on Windows, macOS, Ubuntu, and Android.
- Large-Scale Dataset: Contains data from six operating systems and three task domains.
- Flexible Training: Supports training with Qwen2.5-VL and InternVL.
- Interactive Playground: Enables hands-on experimentation with agents.
- Online Evaluation: Provides benchmarks for standardized agent evaluation.
- Modular Structure: Separate components for evaluation, playground, and training.
- Open-Source Framework: Facilitates community contributions and further research.
ScaleCUA is an active research project with a growing dataset, models, and a comprehensive training framework. Recent activity includes the release of the dataset and models, and ongoing development efforts are evident in the playground and evaluation suites. The project benefits from community engagement and is actively maintained, though it is still evolving.
ScaleCUA benefits researchers and developers interested in building general-purpose computer use agents. It enables the development of agents capable of automating tasks across multiple platforms, addressing a key limitation in the field. This project provides a valuable resource for advancing the state-of-the-art in computer use and AI agent development, simplifying the process of creating robust, cross-platform automation tools.