Tech Kaizen: Uber Cadence

Uber Cadence is a fault-tolerant stateful Workflow Orchestrator. Workflows provide primitives to allow application developers to express complex business logic as code. The underlying platform abstracts scalability, reliability and availability concerns from individual developers/teams. It is open-source, developed by Uber and written in Go language. Define workflows in the code and Cadence will make sure that the whole workflow code will execute no matter what!

Cadence is a distributed, scalable, durable, and highly available orchestration engine developed at Uber Engineering to execute asynchronous long-running business logic in a scalable and resilient way. The unit of work that you can execute with a Cadence client is the Workflow . This is basically a Go function, which describes the main flow, precedence, branching, or, for example, iterations of actions. However there are some rules that you have to follow to get a correct and reliable behavior. Workflow encapsulates the orchestration of activities and child workflows.

The workflow is the implementation of the coordination logic. The Cadence programming framework (aka client library) allows you to write the workflow coordination logic as simple procedural code that uses standard Go data modeling. The client library takes care of the communication between the worker service and the Cadence service, and ensures state persistence between events even in case of worker failures. An activity is the implementation of a particular task in the business logic. Activities are implemented as functions. Data can be passed directly to an activity via function parameters.

What makes Cadence different?

What makes Cadence more than a sophisticated distributed task queue manager? You can define workflows in code and let Cadence handle state, timeouts, history and all the other little necessities. Then, those workflows can be inspected after submission, so you can check their progress and results, or send them external signals as needed.

Cadence fault-oblivious stateful code platform preserves a complete multithreaded application state including thread stacks with local variables across hardware and software failures. It greatly simplifies the coding of complex stateful distributed applications. At the same time, it is scalable and robust enough to power hundreds of critical use cases.

Microservice Orchestration and Saga:

The Saga Pattern is a microservices architectural pattern to implement a transaction that spans multiple services. A saga is a sequence of local transactions. Each service in a saga performs its own transaction and publishes an event. The other services listen to that event and perform the next local transaction.

It is common that some business processes are implemented as multiple microservice calls. And the implementation must guarantee that all of the calls must eventually succeed even with the occurrence of prolonged downstream service failures. In some cases, instead of trying to complete the process by retrying for a long time, compensation rollback logic should be executed. Saga Pattern is one way to standardize on compensation APIs.

Cadence is a perfect fit for such scenarios. It guarantees that workflow code eventually completes, has built-in support for unlimited exponential activity retries and simplifies coding of the compensation logic. It also gives full visibility into the state of each workflow, in contrast to an orchestration based on queues where getting a current status of each individual request is practically impossible.