Predicting Cardiac Function From Echocardiograms

Goal

We will build a model that is capable of identifying the left ventricle (LV) from an apical 4-chamber (A4C) view echocardiogram video. The end goal would be to fully automate the process of LV ejection fraction (EF) calculation for cardiologists.

Customer

For cardiologists in use as a diagnostic aid. The user is assumed to have a good working knowledge of echocardiography and interpreting its results. Therefore, the targeted use case is not a substitute for an expert opinion, but rather an aid in particular in critical care and emergency medicine to speed up cardiac function assessment, enhance the accuracy and consistency of the underlying measurements, as well as against fatigue in overworked physicians which may lead to assessment errors.

Impact

The primary impact would be enabling further research. It would open the door to the automation of many other echocardiogram evaluation processes e.g., right ventricle function, with the end goal of enabling faster diagnoses, emergency diagnosis use cases, and reduction of misdiagnoses.

Data

EchoNet-Dynamic published by Stanford which contains a set of Echocardiogram video files with video metadata and expert LV tracings.

Approach

LV volume tracings will to be converted to binary LV mask images as an image processing step. LV binary mask images along with corresponding grayscale echocardiogram video frames will be used to develop a neural network for LV segmentation. U-Net or Mask R-CNN may provide a baseline, and we will train a vision transformer architecture model. We will try single image based 1-pass, or 2-pass image sequence based and temporal mask smoothness imposing segmentation network alternatives as well. In view of the quite distinct image features governing echocardiography videos, a transfer learning approach may be adopted. LV binary mask images (one for each echocardiography video frame), using Computer Vision (CV) blob analysis techniques, will be transformed into LV volume estimates leading to the generation of 1-dimensional LV volume time series. Exact scale corresponding to real-world, due to the nature of EF calculation formula, is not crucial.

A second neural network will be developed to estimate ESV, EDV, and EF based on the above referenced 1D LV volume time series using thin disk approximations. We will explore a transformer network trained to develop an attention span encompassing more than two consecutive opposite LV volume extremes i.e., those at the systolic and diastolic ends. We will use Colab Pro as our primary development environment.

Results

Model Development - III

Time Series Processing