greg.ruthenbeck.io
Intubation simulator

Interactive Soft Tissue for Surgical Simulation

5 min read researchhapticsgpuc++virtual-realitysimulation

My PhD thesis: an ablatable soft-tissue simulation framework that makes efficient use of parallel hardware, and the medical training simulators built with it.

My thesis was the research and development of new technology for the simulation of surgically interactive virtual tissues. It includes haptic rendering, which makes the tissue into something you can touch and feel, novel medical training simulation applications, and reviews of the related technology, hardware and the research area more generally.

Download thesis as a PDF (from ResearchGate, 4MB). Listed on hgpu.org in October 2011.

A haptic device and the demo laptop
Haptic device and demo laptop.
The tissue simulation framework
Tissue simulation framework.

Abstract

Medical simulation has the potential to revolutionize the training of medical practitioners. Advantages include reduced risk to patients, increased access to rare scenarios and virtually unlimited repeat-ability. However, in order to fulfill its potential, medical simulators require techniques to provide realistic user interaction with the simulated patient. Specifically, compelling real-time simulations that allow the trainee to interact with and modify tissues, as if they were practicing on real patients.

A key challenge when simulating interactive tissue is reducing the computational processing required to simulate the mechanical behavior. One successful method of increasing the visual fidelity of deformable models while limiting the complexity of the mechanical simulation is to bind a coarse mechanical simulation to a more detailed shell mesh. But even with reduced complexity, the processing required for real-time interactive mechanical simulation often limits the fidelity of the medical simulation overall. With recent advances in the programmability and processing power of massively parallel processors such as graphics processing units (GPUs), suitably designed algorithms can achieve significant improvements in performance.

This thesis describes an ablatable soft-tissue simulation framework, a new approach to interactive mechanical simulation for virtual reality (VR) surgical training simulators that makes efficient use of parallel hardware to deliver a realistic and versatile interactive real-time soft tissue simulation for use in medical simulators.

Examiner feedback

In his PhD thesis Mr. Ruthenbeck worked on sophisticated scientific and technical tasks related to efficient surgery simulation frameworks and his thesis comprises contributions to various aspects of this research area.

Mr. Ruthenbeck proceeded very carefully and skillfully in the development of the approaches presented in his thesis. He showed a thorough background in the related research areas.

The presented thesis constitutes a valuable contribution to the field of interactive surgery simulators.

Table of Contents
  • Chapter 1. Introduction
    • 1.1 Thesis Aims
    • 1.2 Thesis Outline
  • Chapter 2. Virtual Reality for Medical Training
    • 2.1 Learning Modalities
    • 2.2 VR Medical Simulations: The State of the Art
  • Chapter 3. Simulator Development Tools
    • 3.1 Software Tools
    • 3.2 Literature Surveys
    • 3.3 Recent Advances in Parallel Computing Hardware
    • 3.4 Conclusion
  • Chapter 4. A New Tissue Simulation Framework
    • 4.1 The Tissue Simulation Framework
    • 4.2 System Overview
  • Chapter 5. Mechanical Simulation
    • 5.1 Background
    • 5.2 Cubic Rotational Mass Springs: A New Approach
    • 5.3 Demonstration
    • 5.4 Extensibility
  • Chapter 6. Interactive Marching Tetrahedra
    • 6.1 A Review of Marching Algorithms
    • 6.2 Marching Tetrahedra
    • 6.3 Interactive Marching Tetrahedra: A New Approach
    • 6.4 Demonstration
  • Chapter 7. Integration of Components
    • 7.1 Deforming the IMT Mesh
    • 7.2 Cuts and the Coupled System
    • 7.3 Collisions
    • 7.4 Demonstration
  • Chapter 8. Haptics
    • 8.1 6DOF Haptics
    • 8.2 Desktop Haptic Devices
    • 8.3 Haptics APIs
    • 8.4 Haptic Rendering
    • 8.5 Haptics in the TSF
    • 8.6 Demonstration
  • Chapter 9. Applications
    • 9.1 An Endoscopic Sinus Surgery Simulator
    • 9.2 ISim: An Endotracheal Intubation Simulator
    • 9.3 A Coblation Tonsillectomy Simulator
  • Chapter 10. Conclusion
    • 10.1 Future Directions
    • 10.2 Final Words
  • Bibliography
  • Appendix A: Mesh Coupler C++ Code Listing
  • Appendix B: An Earlier Version of ISim
    • B.1 Image-based Collision Detection and Deformation
Appendix B: Image-based Collision-detection
Where green and yellow are darkest, there is a collision
Where green and yellow are darkest, there is a collision.

A novel method for collision detection, haptic rendering and deformation, using the intersection of depth buffers (the screen-shot above). This approach basically uses the graphics rendering pipeline to do complex intersection tests (collision detection) instead of conventional collision detection, deformation and haptic rendering.

ISim: An Endotracheal Intubation Simulation

This intubation simulation is one of 3 medical simulators that I built during my PhD (see Chapter 10).

VR Cutable Soft Tissue for Surgical Simulation

Creating virtual soft materials that can be cut in real-time requires several complex components to work together. The graphics processing unit (GPU) was used to make this possible using the CUDA API from Nvidia.

Interactive Volumetric Cutting of Soft Tissue

Another demo of the real-time cut-able virtual tissue system that I developed.

Interactive Cut-able Tissue Simulation – Sinus Surgery

An early sinus surgery simulator prototype that demonstrates one of the cut-able virtual tissue systems that I developed.

I had a blast focusing on building cool new 3D tech for an essential part of tomorrow's medical training! 😉