Disque is an in-memory, distributed message broker designed for building reliable job queues. It addresses the need for a scalable and fault-tolerant system to exchange messages between processes. Disque uses a multi-master architecture and supports various delivery semantics like at-least-once delivery. It is implemented in C as a non-blocking networked server.
Disque offers synchronous replication for high availability, supports at-least-once and at-most-once delivery semantics with configurable retry times, and implements best-effort ordering based on message creation time. Its multi-master design allows for flexible scaling and easy node addition without data loss. Disque provides fine-grained control over job delivery times and can operate in both single-data center and multi-data center environments.
- Synchronous Replication: Ensures high availability with replication to multiple nodes.
- At-Least-Once Delivery: Guarantees message delivery even in failure scenarios with configurable retries.
- Multi-Master Architecture: Enables flexible scaling and easy node addition.
- Fine-Grained Control: Offers configurable delay, retry, and expiry times for each job.
- Stateless Iterators: Provides easy-to-use iterators for managing queues and jobs.
- Easy Cluster Management: Simplifies adding and removing nodes without data loss.
- Optional Persistence: Supports disk persistence for single-data center deployments.
Disque is currently in a release candidate state, indicating a stable but evolving project. While the API is considered stable for the immediate future, ongoing development and bug fixes suggest continued active maintenance. The project boasts a good track record of durability and reliability, with a community actively reporting bugs and sharing experiences. The documentation is comprehensive, though further enhancements may be planned.
Disque benefits developers who need a reliable, scalable message broker for asynchronous communication in distributed systems. It is well-suited for use cases such as task scheduling, event processing, and decoupling application components. By providing guarantees for message delivery and offering flexible configuration options, Disque simplifies the development of robust and resilient applications.