About Plaid

Plaid, Inc. is a fintech unicorn valued at $8 billion that provides a global data network to power the tools millions of people rely on to live healthier financial lives. Their ambition is to facilitate a more inclusive, competitive, and mutually beneficial financial system by simplifying payments, revolutionizing lending, and leading the fight against fraud.

Plaid cut CI build times in half with EngFlow

In their post, Plaid's Developer Efficiency team walks through how they migrated 150+ Go services from a single Dockerfile to Bazel's rules_oci, leveraging ephemeral BUILD files to minimize migration overhead. Along the way, they share tips for:

• Choosing a remote cache: Plaid recognized that Bazel alone was insufficient; success required a secure and economical remote cache architecture that kept traffic private, minimized maintenance, and scaled without prohibitive networking costs.
• Dealing with performance hurdles in Gazelle: While Gazelle automated BUILD file generation, Plaid had to optimize local execution by using prebuilt binaries, refining .bazelignore patterns, and minimizing indexing to overcome laptop-specific I/O bottlenecks and maintain a fast developer experience.
• Minimizing impact on developers' day-to-day workflows: Plaid abstracted the migration through existing devenv interfaces and shadow pipelines, ensuring engineers gained the performance of faster CI and local builds without needing to master Bazel internals.

Plaid's Developer Efficiency team needed to speed up its largest CI pipeline without forcing engineers to relearn how they worked. By migrating more than 150 Go services from a monolithic Dockerfile to Bazel-based per-service image builds, and by deploying EngFlow as a co-managed remote cache inside Plaid's AWS account, Plaid cut CI time by 50%, reduced container image size by 90%, and improved local iteration by up to 5x.

The challenge

By late 2023, Plaid's Go monorepo contained 150+ services and dozens of high-fidelity end-to-end integration tests built around docker-compose. To preserve reproducibility, CI rebuilt affected components on every run. That model had worked for years, but at Plaid's scale it had become a bottleneck: p50 CI duration had reached 20 minutes, p95 had reached 35 minutes, and the shared image build alone could take up to 10 minutes.

The problem was not just that the image was large. The bigger issue was operational: one shared container had become the critical path for the whole monorepo. Plaid could not simply split it into 150 smaller images and hope for the best. Without accurate dependency tracking and aggressive reuse across CI, many smaller images could have been just as slow, or slower, than one large one.

Why change now?

Plaid saw in Bazel a path to a more precise build graph, selective rebuilds, and better cache reuse. The attraction was not Bazel in the abstract. It was the combination of strict dependency tracking, remote caching across CI workers, and more efficient image construction without relying on Docker for every step.

Plaid validated the idea with a single-service proof of concept. One senior engineer with no prior Bazel experience completed that first step in about four weeks, and the results were strong enough to justify scaling the approach across the monorepo.

Why EngFlow?

Plaid concluded early that Bazel alone would not be enough. To beat its existing Docker-and-Go pipeline, it needed a remote cache architecture that was fast, economical, and operationally clean.

EngFlow fit Plaid's requirements because it offered a co-managed model: EngFlow engineers co-managed the cache cluster inside Plaid's AWS sub-account. That let Plaid keep traffic private, reduce maintenance burden on the internal team, and take advantage of AWS savings plans already in place.

Implementation details

Plaid designed the migration to minimize developer disruption. The team duplicated the CI pipeline and ran a shadow version against Bazel-enabled services, which let Plaid measure impact and improve operational maturity before moving any production workloads over.

On the build side, Plaid used Bazel, rules_oci, custom macros, and Gazelle automation to move from one large Dockerfile to per-service container images. The team also extended Gazelle with a custom plugin so new services could become Bazel-buildable with minimal extra work from developers.

On the cache side, Plaid implemented automated certificate management with AWS Private CA, restricted cache writes to CI to reduce cache-poisoning risk, and separated deployment caching from pull-request-driven CI. That architecture let Plaid gain the performance benefits of remote caching without weakening its security posture.

The Rollout

From proof of concept to production migration, the effort took about seven months and was executed by four Developer Efficiency engineers, with relatively limited involvement from the teams that owned the services. By mid-2024, Plaid had migrated all relevant services to Bazel-based builds and moved them into production.

Results of Migrating with EngFlow

The result was much more than faster builds and tests. Plaid changed the operating model of CI for a large Go monorepo. CI times fell by 50%. Container images shrank by 90%. Local build cycles were accelerated by up to 5x. And Plaid achieved those gains while largely preserving the developer experience engineers were already used to.

What this shows

Plaid's migration is a strong example of what happens when build-system modernization is paired with enterprise-grade cache operations. Bazel gave Plaid a better dependency model and better image builds. EngFlow removed the burden of building and operating a private, cost-effective, secure remote cache layer from scratch. The outcome was faster CI, faster local development, and a stronger build foundation without a disruptive workflow reset.