Running Regulated AI Inference for Healthcare on Secure Private Cloud

The Problem

The deal was interrupted at legal review, not because the model underperformed or the price was wrong but because nobody on the vendor side could answer one question with a straight sentence: where does the patient's imaging data live and who can access it?

A European health institute, built up over the past decade to bring AI-assisted diagnostics and administrative tooling to public hospital networks across the region, builds inference products for three different buyers at once. Hospital radiology departments use its triage models to flag urgent CT and MRI scans before a radiologist opens the file, health system clinical teams use its ambient documentation and EHR summarisation tools during patient visits, and payer organisations run its claims adjudication and prior authorisation models on submitted case files, which means three buyer types and three risk profiles are sitting on one shared infrastructure layer underneath. A fourth workload runs in the background across all of it: continuous model monitoring and drift detection, watching every prediction the other three models make for signs of degradation before a regulator or a client notices first.

That shared layer was public cloud and it was starting to fail in three specific ways.

• First, latency: acute triage carries a clinical SLA measured in seconds from scan availability to flag, which leaves the infrastructure a network round-trip budget of well under 50 milliseconds, a budget that round trips to a distant, non-EU region consumed on their own, no matter how well the model itself was optimised.
• Second, tenancy: procurement teams at large national health systems and hospital groups run InfoSec reviews that ask direct questions about shared infrastructure, such as who else is on this hardware, what isolates our data from theirs and what happens if their workload spikes during ours and multi-tenant infrastructure does not have clean answers to those questions, a fact reviewers already know well.
• Third, jurisdiction: GDPR obligations apply across every EU hospital and payer client, the EU AI Act classifies the institute's products as high-risk with its transparency obligations applying from 2 August 2026 regardless, while the high-risk conformity requirements follow the Digital Omnibus timeline adopted in June 2026, namely 2 December 2027 for stand-alone Annex III systems such as the payer-facing prior authorisation models and 2 August 2028 for AI embedded in regulated medical devices, which is where the triage and clinical decision support products sit; certain hospital clients carry additional health-data hosting certification requirements on top of that, and US hospital clients bring HIPAA Security Rule obligations into the same environment. None of this is exotic so much as the baseline for anyone selling AI into hospitals and payers in 2026, a baseline multi-tenant, non-EU infrastructure cannot credibly meet.

The common thread across all three was not a compliance inconvenience but a commercial one, since infrastructure ambiguity was the reason a signed deal sat in legal review for six weeks with no path forward.

The institute had tried fixes before this deal stalled: a dedicated VPC in the right region helped with residency but did nothing for tenancy, a latency-optimised CDN layer shaved milliseconds off delivery but left the underlying compute shared and each fix solved one symptom while leaving the other two exposed, because the actual problem was never a single missing feature but the fact that three separate infrastructure guarantees needed to come from the same environment at the same time.

The Infrastructure Decision

The institute's engineering and legal teams sat down and wrote out what the infrastructure actually had to guarantee, not what would be nice to have.

• A sub-50ms network round-trip budget for acute triage traffic within the region
• Single-tenant isolation for all workloads touching PHI
• EU data residency as a hard procurement requirement, not a preference
• Access logging and auditability that could go directly into an InfoSec review pack
• Deterministic performance, because SLA commitments to hospitals cannot depend on how busy a shared cluster happens to be that day

Public cloud, even the EU regions of it, could not clear all five requirements at once: the latency requirement ruled out anything routed through a shared, distant control plane, the tenancy requirement ruled out shared hardware regardless of region, and what the institute needed instead was dedicated infrastructure, deployed where its clients' data actually had to sit, with a support model built for regulated operators rather than self-serve developers.

That is what led the team to Hyperstack Secure Private Cloud, specifically the Dedicated Cloud deployment model, which is the fully managed version of Secure Private Cloud: Hyperstack owns everything from power and cooling through the orchestration layer, while the institute retains control of its models, application logic, and data governance, and because the model is not self-serve, the environment is designed, built and validated against the institute's actual requirements before go-live, running on single-tenant, segregated infrastructure with no shared-tenancy exposure to answer for. Secure Private Cloud is commissioned at cluster scale on twelve-month minimum terms, which meant the institute was sizing its architecture as a long-term platform decision rather than a short-term capacity fix. Region selection was part of the same conversation, since the institute's EU workloads sit in EU data centres chosen for regulatory alignment and where certain clients required it, the team could point to sovereign build positioning that reduces exposure to the CLOUD Act without overstating what that guarantees on its own.

The Dedicated Cloud deployment model also let the institute allocate hardware dynamically through the portal and APIs across virtual machines, containers and Kubernetes clusters, which mattered because the institute was not running one workload but four, each with a different latency and compliance profile on the same deployment.

Before any production traffic touched the new environment, the institute ran the deployment through acceptance testing against its own workloads, real triage images, real documentation samples and real claims files, measured against the latency and throughput numbers the team had written down at the start, a step that mattered as much as the hardware selection itself, because a deployment that looks right on paper still has to prove itself against what the vendor actually runs before anyone stakes a hospital SLA on it.

The Build

The four inference workloads were not built identically, since each one got infrastructure matched to what it actually needed, ordered here by latency sensitivity.

Medical imaging triage inference

Covering CT, MRI, and whole-slide pathology, this workload runs on GPU-enabled Kubernetes with RoCE fabric (RDMA over Ethernet), with local NVMe handling active inference that needs fast, ephemeral scratch space while Shared Storage Volumes persist model artefacts across restarts. Whole-slide pathology models process multi-gigapixel slide images, so NVIDIA ConnectX-8 SuperNICs are used where moving slide tiles into GPU memory becomes the bottleneck rather than compute itself, and where those same datasets need shared, high-throughput file access across multiple nodes at once, a parallel filesystem sits underneath the workload rather than forcing everything through block storage built for a single machine.

Clinical NLP inference

Covering ambient documentation and EHR summarisation, this workload also runs on Kubernetes with RoCE fabric, with SSVs holding model weights so they persist across node changes, and EU region selection was non-negotiable here since this workload processes patient conversation data directly and residency requirements apply from ingestion onward.

Prior authorisation and claims adjudication

This workload runs on the same Kubernetes and RoCE combination, but with a heavier emphasis on SSVs for persistence and access logging configured specifically for payer compliance reviews, since claims data carries its own audit trail requirements separate from clinical data, and the deployment reflects that distinction.

Continuous model monitoring and drift detection

This workload runs on lightweight GPU allocation, since it does not need the throughput the other three demand, while Secure Object Storage holds inference logs and drift metrics long term, giving the institute's data science team a durable record for model governance and for the EU AI Act conformity documentation due ahead of the Act's 2027 to 2028 high-risk deadlines.

Four workloads sit inside one single-tenant environment, each with a different infrastructure profile, and that modularity is the point of Dedicated Cloud: the architecture matches how the institute actually operates rather than a generic template it has to work around.

What Changed

Network round-trip latency for acute triage came down into single-digit milliseconds for in-region clients, leaving nearly the full latency budget for model compute, because the round trip to a distant shared region was removed from the equation entirely. End-to-end triage flags now land comfortably inside the clinical turnaround SLA, fast enough for the workflow to actually work the way it was designed to.

InfoSec and procurement reviews changed shape: instead of explaining what protects the institute's clients from other tenants on shared hardware, the conversation became a walkthrough of a single-tenant environment with documented access controls and audit trails, giving reviewers something concrete to sign off on rather than a set of assurances to accept on faith.

Deal velocity through legal and compliance improved the most, since the question that stalled the original deal (where does the data live and who can access it) now has a documented answer before a client ever asks it, which moved procurement conversations from open-ended risk assessments to checklist confirmations. Cost planning also became more predictable, because dedicated, non-oversubscribed resources meant the institute's finance team could forecast infrastructure spend against contracted capacity instead of against usage spikes it could not fully control on shared hardware.

The Operational Layer

Inference pipelines for triage, documentation, and claims run continuously rather than on a nine-to-five schedule, and since clinical SLAs do not pause overnight, the institute's deployment includes Hyperstack's 24/7/365 operations model with continuous monitoring through a dedicated Network and Operations Centre and follow-the-sun coverage across regions.

Incident response follows a tiered escalation path from Ops through Technical Engineering to Infrastructure Engineering, and Severity 1 incidents, the category a production triage pipeline outage falls into, carry a target response time of 30 minutes and a target resolution within 4 hours, a commitment that is not a minor line item for a vendor whose hospital clients measure downtime in delayed diagnoses rather than lost revenue.

The institute also gets named support roles rather than a ticket queue, including a Technical Customer Success Manager coordinating delivery and escalations, 24/7 support engineering for monitoring and incident response, and a Machine Learning Engineer during onboarding for workload migration and benchmarking, while maintenance governance adds predictability on top of that: scheduled maintenance requires at least 14 days' notice, and emergency maintenance is limited to critical fixes with immediate communication, which matters for a vendor running four production pipelines against hospital and payer SLAs simultaneously. Because the deployment sits under the Dedicated Cloud model, availability is a contractual number rather than a best-effort promise: platform-level targets covering infrastructure and scheduling operations are defined at contract, typically 99.9% or above depending on design, which gives the institute's own leadership something concrete to plan capacity and client commitments against.

None of this replaces the institute's own operational ownership of its models and applications; it replaces the uncertainty that used to sit underneath them.

What This Means For Similar Deployments

This institute's situation is not unusual for healthcare AI vendors operating across multiple regulatory regimes and buyer types at once, since the pattern repeats: infrastructure ambiguity survives as a technical footnote until it becomes the reason a deal cannot close. Single-tenant, dedicated infrastructure does not just solve a latency or compliance problem in isolation; it gives sales and legal teams a documented, defensible answer to the question every serious enterprise buyer eventually asks, one that does not need to be perfect or exhaustive but needs to exist in writing before the buyer's InfoSec team asks for it, because the alternative is a deal sitting in review with no clear owner and no clear timeline.

Book a Scoping Call

If your team is running multiple regulated AI workloads on shared infrastructure and finding that the conversation with InfoSec or procurement keeps stalling in the same place, it is worth a direct conversation about what a dedicated environment would actually look like for your specific workloads. Book a scoping call with the Hyperstack team to talk through a Secure Private Cloud deployment built around your latency, residency, and audit requirements.

FAQs

Why was the public cloud deployment no longer suitable?

Shared infrastructure created latency, tenancy and data residency concerns, preventing legal approval despite strong model performance and competitive pricing.

Why did the institute choose Hyperstack Secure Private Cloud?

Hyperstack Secure Private Cloud provided dedicated single-tenant infrastructure, EU residency, predictable performance, managed operations and documented controls supporting regulated healthcare deployments.

What workloads ran on the Secure Private Cloud?

The environment supported medical imaging triage, clinical NLP, claims adjudication and continuous model monitoring, each with tailored infrastructure profiles.

How did Dedicated Cloud improve compliance readiness?

Single-tenant infrastructure, audit logging, EU residency and documented operational controls simplified procurement reviews and strengthened regulatory confidence significantly.

Why were separate infrastructure profiles used for each workload?

Each application had unique latency, storage, compliance and performance requirements, making customised infrastructure more efficient than one standard deployment.