Transforming Endoscopic Documentation: Novel Artificial Intelligence Summarization Techniques for Pouchoscopy Videos

Background:
Endoscopy is crucial in the evaluation and management of patients with Inflammatory Bowel Disease. Managing the patient clinically relies on endoscopic images, often performed by a different gastroenterologist. Ulcerative colitis is a chronic inflammatory bowel disease which may require surgical management including creation of an IPAA (ileal pouch anal anastomosis or J pouch). There are many potential surgical and inflammatory conditions which can involve the J pouch and require close management by IBD specialists. We postulate that having access to endoscopic videos will improve understanding of the complex anatomy and pathology encountered during pouchoscopy.
Methods:
Our GI division has a large repertoire of endoscopic videos. They are reviewed for patient care and serve as a valuable resource for medical education and training. However, the sheer volume and length of procedural videos presents challenges for information retrieval and consumption. Summarizing these videos by retaining only relevant information offers a concise overview, making the content more accessible and manageable. We developed tools for video summarization, including key frame extraction and video skimming using computer vision techniques. We employ existing endoscopic images to automatically summarize videos. Short clips are created around documented pictures and merged to form a comprehensive video, with anatomical locations titled for each segment. Chapters are added for easy navigation.
Results:
Following our methodology, 10 endoscopy videos were summarized. The mean length of original videos was 10.26 (5.22-16.21) minutes and summarized videos was 2.54 (1.34-4.02) minutes. The original and summarized videos were reviewed by 2 experienced IBD endoscopists to assess accuracy and veracity.
Conclusions:
This shift from text-based to video-based documentation in GI procedures represents a significant paradigm change. The summarized videos are available for physicians and medical care teams for documentation, follow-up care, or collaboration. Our method is generalizable to other procedures and departments. Effective video summarization enhances user experience by providing quick overviews, aiding in decision-making, and saving time.