Logo image
Predicting future surgical steps during MCA aneurysm clipping using a multimodal transformer
Journal article   Open access   Peer reviewed

Predicting future surgical steps during MCA aneurysm clipping using a multimodal transformer

Thomas J. On, Jonathan A. Tangsrivimol, Jonathan A. Tangsrivimol, Jiuxu Chen, Yuan Xu, Baoxin Li, Michael T. Lawton and Mark C. Preul
Frontiers in surgery, Vol.13
06/01/2026

Abstract

anticipating surgical steps artificial intelligence middle cerebral artery aneurysm neurosurgical education predicting surgical steps surgical video
ObjectiveIntracranial aneurysm clipping is technically demanding, with dynamic anatomy and evolving intraoperative decisions. Although AI has been applied to retrospective surgical phase recognition, near-future surgical step prediction remains largely unexplored in neurosurgery. This study evaluated the feasibility of fixed-horizon surgical step prediction in recorded microscope videos of middle cerebral artery aneurysm clipping operations.MethodsWe retrospectively analyzed 25 uncomplicated MCA bifurcation aneurysm clipping surgeries by a single neurosurgeon, using 18 for training and 7 for independent testing. Cases were annotated into 12 standardized operative steps. A transformer-based prediction framework was evaluated with three input configurations: video features alone, prior human-annotated step labels alone, and combined video plus step-label inputs. With a sliding-window approach, each configuration used 1 min of input data to predict the operative step labels occurring during the subsequent 1 min. Performance was assessed by accuracy, weighted F1 score, and sequence-level agreement with ground truth.ResultsThe multimodal model achieved the highest mean accuracy and weighted F1 score, 0.683 and 0.673, compared with 0.606 and 0.577 for the annotation-only model and 0.477 and 0.447 for the video-only model. The multimodal model also showed the best sequence-level alignment, with a normalized edit distance of 0.430 and edit score of 0.570.ConclusionFixed-horizon surgical step prediction during MCA aneurysm clipping was feasible under controlled input conditions. Multimodal modeling provided the strongest predictive performance. These findings represent upper-bound performance and require validation in fully automated recognition-to-prediction pipelines.
url
https://doi.org/10.3389/fsurg.2026.1827725View
Published (Version of record) Open

Metrics

1 Record Views

Details

Logo image