Real-Time AI with NVIDIA Holoscan

Accelerating AI-Powered Medical Imaging

Artificial Intelligence is transforming how medical devices acquire, process, and interpret visual information.

In operating rooms (OR), endoscopy labs, and clinical-edge deployments, modern systems increasingly combine real-time video processing with AI inference to support decision-making and patient outcomes.

In these environments, performance is not only measured in raw throughput. It is also measured in end-to-end responsiveness: how quickly data can move from sensor to compute, how efficiently inference can run, and how reliably results can be rendered or transmitted for human or machine action.

NVIDIA Holoscan is an AI sensor processing platform for low-latency streaming workflows across embedded, edge, and cloud deployments, with its primary design focus on medical device development. DELTACAST contributes Holoscan-compatible modules for its video capture and processing boards through an open-source repository, helping teams connect professional video capture and output into Holoscan application pipelines.

Medical AI at the Core of Holoscan

NVIDIA Holoscan was originally conceived as an AI sensor processing platform with its primary application domain in medical device development. Clinical and surgical environments probably impose the most demanding requirements on real-time video AI systems: deterministic latency, high image fidelity, and the ability to overlay AI-generated guidance on live feeds without interrupting the clinical workflow.

A common operator-based pipeline model reduces redevelopment effort across projects. Teams can apply consistent architectural patterns when moving from proof-of-concept to production or from a research prototype to a regulatory-grade medical device deployment.

Medical Imaging

NVIDIA showcases medical and clinical-edge workflows such as surgical video AI pipelines, segmentation, and visualization in Holoscan reference applications.

For clinical and medical-device contexts, real-time operation can directly impact usability and patient outcomes. Holoscan reference workflows illustrate how multi-stage pipelines can ingest live video, run AI inference, and present contextual output with minimal delay, a critical requirement in operating rooms and interventional suites where every millisecond of added latency has practical consequences.

AI-driven Computer-Assisted Decision (CAD) and Computer-Assisted Surgery (CAS) systems built on Holoscan positively impact patient outcomes by delivering timely and accurate visual guidance to practitioners. DELTACAST and NVIDIA have been collaborating since the inception of the Holoscan program on this exact challenge: combining low-latency AI inference with live surgical video in a single, reliable pipeline.

 

 Typical medical applications include:

• Endoscopy and minimally invasive surgical workflows
• AI-assisted surgical video processing and instrument tracking
• Real-time tissue segmentation and anatomical structure identification
• Surgical navigation and augmented-reality guidance overlays
• Medical imaging inference pipelines (ultrasound, colonoscopy, laparoscopy)
• Computer-Assisted Surgery (CAS) visualization
• Real-time AI-generated annotations composited onto live video

In practical terms, these patterns support rapid experimentation with model updates, preprocessing stages, postprocessing logic, and visualization flow without requiring teams to re-architect the full application each time.

Industrial and Edge AI

Holoscan is a domain-agnostic platform, and the same pipeline model that serves medical imaging also applies to industrial inspection, robotics, and edge AI workloads — machine vision, sensor fusion, and streaming analytics. These use cases benefit from the same low-latency architecture but are not the primary focus of this document.

NVIDIA Holoscan Platform

NVIDIA describes Holoscan as an AI sensor processing platform combining hardware systems, optimized libraries, and software components for low-latency sensor and network connectivity.

At a platform level, this means developers can build around an architecture where high-bandwidth sensor data, AI models, and streaming operators are treated as parts of one integrated system rather than disconnected building blocks.

NVIDIA IGX

NVIDIA IGX is NVIDIA's industrial-grade, enterprise-ready edge AI platform for industrial, robotics, and medical environments, with emphasis on real-time sensor processing, safety, and long lifecycle support options. It is the primary deployment target for Holoscan-based medical device applications.

For medical device manufacturers, IGX is positioned as a deployment foundation when enterprise supportability, functional safety considerations, and production-readiness matter alongside raw AI capability. It can be embedded directly in a medical device or attached as a clinical-edge sidecar.

The IGX platform has evolved across two generations:

NVIDIA IGX Orin :

The first IGX generation, built around the NVIDIA Orin SoC (Ampere GPU architecture). Available in production form as the IGX Orin 700 (MicroATX board kit with dGPU support, ConnectX-7 networking, functional safety island, and BMC) and as a developer kit for early adoption and prototyping.

NVIDIA IGX Thor :

The current generation, powered by the NVIDIA Blackwell architecture iGPU and an optional discrete GPU, delivering up to 5581 FP4 TFLOPS of AI compute. IGX Thor is purpose-built for physical AI and advanced sensor processing workloads.

The product family includes:

• IGX T7000 : MicroATX board kit with dGPU support, functional safety, NVIDIA ConnectX-7, and BMC support. Its MicroATX form factor includes PCIe expansion slots, making it directly compatible with PCIe add-in cards such as DELTACAST video capture and processing boards
• IGX T5000 : System-on-Module for custom carrier board designs, with optional functional safety features
• IGX Thor Developer Kit : Full development kit with a pre-installed dGPU, suitable for all IGX Thor use cases including Holoscan pipeline development
• IGX Thor Developer Kit Mini : Compact development kit focused on robotic safety use cases

The MicroATX board designs (T7000, Orin 700) are particularly relevant for DELTACAST integration: their PCIe slots accommodate DELTACAST SDI and HDMI capture and keyer cards, enabling direct low-latency video IO within the IGX system without external cabling or separate chassis.

NVIDIA Jetson Platforms

NVIDIA Jetson modules are described as compact, power-efficient edge AI modules with JetPack software support and a lineup spanning embedded and robotics use cases.

Jetson is a strong fit for constrained form factors and power envelopes (including certain medical device housings) where edge inference must be delivered in compact, thermally managed footprints.

NVIDIA RTX Workstations and Servers

For workstation and data-center-class visual computing and AI workflows, NVIDIA presents RTX workstation platforms across desktops, laptops, and data center GPUs.

These environments can be valuable for development, validation, simulation, and high-performance inference and visualization workloads that complement medical device edge deployment targets.

NVIDIA Holoscan SDK

The Holoscan SDK provides installation paths (container, Debian package, Python wheel, source build), C++ and Python APIs, built-in operators, examples, and integration with the broader Holoscan ecosystem.

This flexibility is important because medical devices engineering teams tend to adopt different development and deployment strategies. Some prioritize containerized workflows; others prefer system packages or Python-first iteration. Holoscan supports these different paths while maintaining a common programming model.

A Framework for Real-Time Processing

Holoscan SDK is designed for streaming AI pipelines and multimodal sensor-processing workflows, including IO, inference, processing, and visualization operators.

Applications are composed from operators connected into dataflow graphs. This supports clear separation of concerns, where each stage handles a distinct responsibility and can be tuned, replaced, or extended as requirements evolve.

Key Advantages

• Low-latency streaming pipeline architecture
• GPU-accelerated processing
• C++ and Python API support
• Built-in and reusable operators
• Multiple packaging/deployment options (including containers)
• Scalability from embedded to edge/cloud scenarios

Collectively, these capabilities help medical device teams move from prototype to deployable system while preserving architectural consistency.

HoloHub Ecosystem

NVIDIA HoloHub is an open-source repository for reusable operators, modules, tutorials, benchmarks, and end-to-end sample applications.

HoloHub complements the SDK by providing practical implementations that can accelerate onboarding and reduce initial integration effort. Instead of starting from a blank slate, teams can inspect working patterns for acquisition, inference, transport, and rendering.

HoloHub helps teams:

• Explore reference applications
• Reuse operators and modules
• Evaluate practical workflows
• Learn implementation patterns

It also serves as a community exchange layer where reusable components can be shared and adapted for domain-specific needs, including medical device applications.

DELTACAST Holoscan Modules

DELTACAST publishes a Holoscan module repository that extends Holoscan with operators for DELTACAST VideoMaster capture card workflows.

This provides a direct path for teams using DELTACAST hardware to integrate professional video IO inside standard Holoscan application graphs. See the DELTACAST Holoscan integration page and DELTACAST's GitHub organization for the full ecosystem of open-source components.

In practical terms, the project follows the Holoscan Module model: a reusable, distributable package that can be consumed by downstream Holoscan applications rather than copied as project-specific code. This is important for maintainability because teams can keep video capture and transmit logic versioned as a dependency.

Native VideoMaster Integration

The repository documents two primary operators:

VideoMasterSourceOp

Captures from a DELTACAST VideoMaster card.

The source operator is intended for ingesting live streams into a Holoscan graph, where downstream operators can run preprocessing, inference, postprocessing, and visualization.

VideoMasterTransmitterOp

Transmits to a DELTACAST VideoMaster card.

The transmitter operator is used to return processed frames to external video infrastructure for monitoring, routing, or recording.

Together, these operators let developers close the loop from acquisition to output in a single low-latency pipeline.

Operator Capabilities and Configuration

Both operators are exposed in C++ and Python, which supports mixed development workflows:

• C++ for performance-critical and production-facing components
• Python for rapid iteration and experimentation

The documented parameter model includes hardware and stream controls such as:

• Board and channel selection
• Stream dimensions (width and height)
• Progressive/interlaced mode
• Frame rate
• RDMA toggle (when available in the driver/runtime stack)

For output workflows, the transmitter configuration also includes an overlay-related control for card-provided overlay behavior.

This parameterization makes it possible to adapt one operator set across multiple deployment configurations without changing core pipeline structure.

Hardware Keyer Cards for Live Medical Overlay

A distinctive capability of certain DELTACAST cards is the onboard hardware keyer. In a Holoscan medical imaging context, this is particularly relevant: the AI pipeline generates an overlay — segmentation masks, guidance annotations, anatomical markers — and the keyer card composites that overlay onto the live surgical or endoscopy video feed entirely in hardware.

This matters in the operating room because:

• End-to-end latency is shortened to a few dozen pixels : background video passes through the card with near-zero delay, while the PC handles only the overlay graphics
• PCIe bandwidth and CPU cycles are spared : moving a full 4K60 background frame through the host is avoided entirely
• Failsafe bypass relays ensure the live video feed continues to the display even in the event of a power outage, application crash, or system failure — a hard requirement in clinical environments

DELTACAST and NVIDIA have been collaborating since the inception of the Holoscan program to offer a unique solution: AI-generated overlays composited on live SDI camera video in 4K, secured through exclusive hardware bypass.

DELTACAST cards with onboard hardware keyer include:

Part number	Max format	Inputs	Outputs	Keyers	Bypass relays
DELTA-12G-elp-key 11	4K60	1	1	1	Yes
DELTA-12G11-hmi11-e-key	4K60	2	2	1	Yes

 

The DELTA-12G-elp-key 11 is particularly well-suited for compact surgical video AI integrations requiring 4K60 SDI with a single keyer channel and hardware bypass. For mixed SDI/HDMI input scenarios — common when integrating multiple camera types in a procedure room — the DELTA-12G11-hmi11-e-key offers 4K60 keying across both interfaces.

For more detail on the keying technology, see the DELTACAST Keyer technology page.

GPUDirect

NVIDIA GPUDirect is defined as a family of technologies that improves data movement and access for GPUs by enabling direct transfer paths between GPU memory and devices such as network adapters and storage.

NVIDIA states these paths can eliminate unnecessary memory copies, decrease CPU overhead, and reduce latency.

In end-to-end streaming systems, these characteristics can materially improve responsiveness and throughput — particularly in surgical and endoscopy AI pipelines, where minimising latency between camera capture and on-screen overlay presentation directly affects the practitioner's experience. DELTACAST cards support GPUDirect RDMA, enabling peer-to-peer transfers between DELTACAST I/O cards and the GPU without passing through host memory.

GPUDirect includes multiple technologies (such as RDMA, Storage, P2P, and Video), each targeting different data-path scenarios.

Build and Deployment Paths

The module is designed for two common usage paths:

• HoloHub CLI flow for containerized build-and-run workflows
• Direct CMake flow for local and external project integration

The project also provides a hardware-independent mock SDK mode by default, allowing compile, smoke tests, and basic pipeline validation in environments without physical capture hardware.

When teams are ready to target real hardware, the module can be pointed to the VideoMaster SDK installation path and switched from mock mode to hardware mode through build configuration.

This dual-mode approach is valuable for CI and early development because it decouples software progress from lab hardware availability.

Open and Reusable Architecture

The DELTACAST Holoscan module is documented as:

• Open-source (Apache-2.0)
• A Holoscan Module package
• Usable with C++ and Python pipelines

The operator implementation is organized around a dedicated VideoMaster operator subtree and associated Python bindings, aligning with Holoscan module conventions for reusable packaging.

In the broader HoloHub ecosystem, this module can be consumed as a dependency by downstream applications, enabling teams to keep acquisition and transmission concerns modular while composing domain-specific inference and visualization stacks on top.

Documented testing currently emphasizes importability and build-smoke validation, with hardware-backed live-pipeline validation tied to availability of compatible DELTACAST boards and runtime setup.

Example Applications

Medical Imaging and Clinical Edge

Medical applications are the primary use case for the DELTACAST and Holoscan combination.

Key scenarios include:

• Endoscopic AI guidance : live AI inference on endoscopy or laparoscopy video with real-time overlay of anatomical annotations, polyp detection highlights, or navigation cues
• Surgical video AI pipelines : multi-stage Holoscan graphs ingesting camera feed, running segmentation or instrument tracking, and outputting composited video to the surgeon's display
• Medical segmentation workflows : real-time delineation of tissue boundaries, lesion markers, or surgical margin guidance
• Computer-Assisted Surgery (CAS) visualization : augmented views combining live video with AI-generated spatial guidance
• Clinical-edge inference : on-device AI processing on NVIDIA IGX or Jetson platforms embedded in or alongside medical equipment

The DELTACAST hardware keyer cards are especially relevant here: they enable AI-generated overlays to be composited onto live surgical video at near-zero latency, with hardware-guaranteed failsafe bypass meeting the operational requirements of the OR environment.

Industrial and Edge AI

The same Holoscan platform and DELTACAST operator model also support industrial inspection, robotics perception, and sensor-stream processing workloads, for teams that share infrastructure with medical projects or are developing multi-domain AI solutions.

Research and Development

• Reference pipelines for experimentation and benchmarking
• Modular operator and application prototyping
• Rapid hypothesis testing using production-relevant architectural patterns

Build the Next Generation of Real-Time Medical AI Applications

NVIDIA Holoscan provides the platform and SDK foundation for real-time AI sensor processing, with a design centre in clinical and medical device applications. DELTACAST Holoscan modules add VideoMaster-specific capture and transmit operators — and, uniquely, hardware keyer cards that composite AI-generated overlays onto live surgical video with near-zero latency and hardware failsafe protection.

Together, these components help medical device teams design systems that ingest live surgical or endoscopy video, run AI inference, and deliver annotated output to the practitioner with minimal delay and maximum reliability.

The overall value is architectural continuity: a shared development model that scales from experimentation to production deployment, with clear integration points for specialised IO and overlay hardware.