Open Benchmarks Just Sparked a FHIR Vendor Response

Open Benchmarks Just Sparked a FHIR Vendor Response

A new open-source FHIR server benchmark from Health Samurai went live on June 29, 2026, comparing Aidbox, HAPI FHIR, Medplum, and the Microsoft FHIR Server on the same hardware, with the dashboard rerunning daily. Within hours, the Medplum CTO publicly forked the benchmark repository. That fork is the most interesting part of the news, because it shows the competitive dynamic that open, reproducible benchmarks are supposed to create.

It is worth noting up front that the benchmark is vendor-run; Health Samurai builds Aidbox. The interesting move is not the leaderboard but the response.

What the Fork Actually Means

When a competing vendor forks an open benchmark repo, the practical read is straightforward: the Medplum engineering team now has a reproducible harness they can run privately, tune against, and submit improvements back through. That is healthier than the alternative, which is closed-door benchmarks that no third party can replicate.

The numbers under discussion are the ones the Medplum team will optimise against. From the 2026-06-29 snapshot, Medplum sits at about 1,420 CRUD requests per second, 764 resources per second on bundle import, and 1,796 RPS on FHIR search. The search number is worth flagging on its own, because it puts Medplum second across the four servers tested, ahead of HAPI's 1,005 RPS. The benchmark notes that composite search is not supported on Medplum, which is one of the gaps an optimisation pass would naturally close.

The trade-off for the Medplum side: bundle import at 764 res/sec is well behind Aidbox's 2,678 and HAPI's 2,214, which is the kind of number an active community would want to move. The fork makes that work visible.

Why Open Benchmarks Change Vendor Incentives

The vendor response pattern that an open benchmark unlocks is different from the marketing-deck pattern. In a closed comparison, the only people who can verify a number are the people running the test. In an open repository with a daily rerun dashboard, every claim is reproducible by a third party, which puts pressure on every vendor in the comparison to make their numbers honest and their performance better.

For healthcare IT teams evaluating FHIR servers for any workflow, from wellness coaching backends to physical therapy outcomes pipelines, this shift matters. It means the next round of vendor selection conversations can reference numbers that did not exist a month ago. The 6 FHIR servers that handle wellness coaching data cleanly walkthrough covers an adjacent comparison pattern that benefits from the same dynamic. The FHIR product comparison index holds the broader reference set, and the open-source vs commercial FHIR servers for wellness-focused clinics writeup is the natural follow-up if the open-vs-closed question is the active one for your team.

What to Watch Next

The interesting outcome from this fork is not whether Medplum's next release closes the bundle import gap. It is whether the other vendors in the comparison respond in kind. HAPI, the Microsoft FHIR Server, and Aidbox itself all have public room to optimise against the harness, and the daily-rerun dashboard makes every change visible. That is the operating model an open benchmark is supposed to create, and the Medplum fork is the first proof point that it works.

The benchmark covers a 1,000-patient Synthea dataset, roughly 2 million resources, which fits in memory. The next round of public numbers, when the dataset grows past that, is the one where the vendor responses will get most interesting.