PlanOut is a multi-platform framework and programming language designed for online field experimentation. It allows for easy iteration on experiments while managing constraints of deployed online services. Developers define experiments by mapping user IDs to different conditions, leveraging a domain-specific language and a reference implementation in Python. PlanOut handles the randomization process, hashing user IDs to ensure consistent assignments for each experiment.
PlanOut provides extensible classes for defining experiments, supporting multiple languages like Java, JavaScript, and PHP, and includes an interactive web-based editor. It is notable for its deterministic random assignment, automatic logging of experiment data, and its compatibility with large-scale production environments exemplified by Facebook's usage. The framework offers a comprehensive toolset for designing and running controlled experiments.
- Experiment Definition: Extensible classes for defining and implementing complex experiment logic.
- Multi-Language Support: Implementations available in Python, Java, JavaScript, PHP, Go, Julia, and Ruby.
- Deterministic Assignment: Consistent random assignment of users to experimental conditions.
- Web Editor: Interactive web-based editor and compiler for developing and testing PlanOut scripts.
- Data Logging: Automatic logging of experiment data for analysis and reporting.
- Namespaces: Support for managing multiple mutually exclusive experiments.
- Scalability: Designed for large-scale deployments and handling hundreds of millions of users.
PlanOut is a mature framework with a reference implementation in Python and implementations in several other languages. It has been used in large-scale experiments and benefits from a published research paper. Recent activity is indicated by ongoing development of implementations in Go, Julia, and Ruby. Documentation is comprehensive, including installation instructions and a detailed design paper.
PlanOut benefits researchers, students, and businesses seeking a robust framework for conducting online experiments. It enables A/B testing, controlled trials, and data-driven decision-making by providing a scalable and reliable platform. Compared to manual experimentation or less structured tools, PlanOut streamlines the experimentation process, improves data integrity, and facilitates reproducibility.