neural-backed-decision-trees explores a novel approach to enhance the interpretability of deep neural networks. NBDTs combine neural networks with decision trees, iteratively refining the network's structure based on decision tree principles. This framework improves both the accuracy and explainability of predictions, particularly on image classification tasks like CIFAR10, CIFAR100, TinyImagenet200, and ImageNet. By leveraging the strengths of both architectures, NBDTs achieve competitive performance while providing human-understandable decision paths.
This project achieves state-of-the-art results on benchmark image classification datasets, often surpassing standard neural networks in accuracy and generalization. The core innovation is the integration of a decision tree-based loss function during neural network training, effectively learning decision boundaries. The repository offers easy-to-use implementations, pre-trained models for various architectures (ResNet, WideResNet, EfficientNet), and clear instructions for converting existing neural networks. The framework extends applicable to different datasets & model architectures.
- Model Conversion: Provides methods for converting existing neural networks to NBDTs by incorporating a soft tree supervision loss mechanism.
- Pretrained Models: Includes pre-trained NBDT models for ResNet, WideResNet, and EfficientNet architectures on CIFAR10, CIFAR100, and ImageNet datasets.
- Flexible Loss Integration: Allows integration with various loss functions through the
SoftTreeclass, enabling fine-grained control over the NBDT training process. - Extensibility: Supports various neural network models and dataset configurations, allowing researchers and practitioners to easily adapt the framework to their own projects.
- CLI Tool: Offers a command-line interface for quick inference and model evaluation.
- Python Implementation: Written in Python and well-documented with examples and tutorials.
- Colab Notebooks: Provides interactive Google Colab notebooks for easy experimentation and model deployment.
The project is actively developed and maintained with recent updates including arXiv publication and ICLR 2021 acceptance. The repository has a strong community following (624 stars, 128 forks) and comprehensive documentation with clear examples which lead to increased adoption. Regular commits and proactive community engagement suggest healthy project maintenance.
neural-backed-decision-trees benefits researchers and practitioners seeking to improve the interpretability and performance of deep learning models. It is particularly valuable for applications requiring high accuracy and explainability, such as image classification and computer vision. By providing a practical and well-documented framework for integrating decision trees into neural networks, it reduces the barrier to entry for those interested in building more transparent and trustworthy AI systems. This offers a powerful alternative or complement to traditional interpretability methods like attention visualization.