crepes implements conformal prediction methods for classifiers, regressors, and predictive systems. It builds upon existing machine learning models from libraries like scikit-learn to provide well-calibrated p-values or prediction sets. The core problem addressed is the need for methods to quantify uncertainty in predictions, offering not just point predictions but also confidence intervals or sets of plausible outcomes. The library offers both standard and Mondrian conformal approaches, along with tools for assessing the assumptions underlying conformal prediction, such as exchangeability.
crepes offers flexibility by allowing users to integrate their own difficulty estimates and non-conformity scores. It provides both standard and Mondrian conformal classifiers and regressors, catering to different theoretical frameworks. The ability to implement semi-online methods allows for updating predictions as new data becomes available. The code is well-documented, and examples are provided, facilitating easy adoption.
- Classifiers: Implements standard and Mondrian conformal classifiers for classification tasks.
- Regressors: Offers standard, normalized, and Mondrian conformal regressors for regression problems.
- Predictive Systems: Provides tools for creating conformal predictive systems.
- Flexibility: Allows integration with own difficulty estimates, non-conformity scores, and Mondrian categorizers.
- Online Capabilities: Supports semi-online conformal prediction for updating predictions with new data.
- Evaluation: Includes functionalities for evaluating conformal predictions with various metrics.
- Mondrian Support: Provides options for implementing Mondrian conformal prediction with different categorizers.
The crepes package is actively maintained with recent commits and a growing number of stars and forks, suggesting ongoing development and community interest. Comprehensive documentation is available, and examples are provided, indicating a reasonable level of maturity. Regular release updates indicate active maintenance and bug fixes.
crepes is beneficial for applications requiring reliable uncertainty quantification in machine learning models. It's particularly useful in scenarios where knowing the confidence or range of possible outcomes is critical, such as risk assessment, medical diagnosis, and financial modeling. It provides a valuable tool beyond point predictions, allowing for more informed decision-making by offering prediction sets and intervals with guaranteed coverage.