Large Language Models Enable Multi-modality Integration for Brain Tumor Diagnosis and Prognosis

Brain tumors, particularly gliomas, remain challenging to diagnose and treat. While radiology reports can offer non-invasive insights into tumor size, shape, and progression, with pathology reports providing definitive evidence of cellular and molecular features, the lack of integration between these two information sources often complicates clinical decision-making.

To overcome this, a research team led by Dr. Zhuoqi Ma from the Department of Radiology at Brown University and Brown University Health created a pipeline that employs a pre-trained large language model to unify radiology and pathology reports.

In a sample of 426 patients, the system achieved a micro F1-score of 0.849 for tumor presence and 0.929 for tumor stability, surpassing single-source methods by more than 10%.

“Large language models  (LLM) can synthesize information from multiple domains and deliver a more complete picture of tumor status,” shares Ma. “This not only improves diagnostic accuracy but also allows the model to predict survival outcomes without additional training.”

The team further validated their approach in an independent cohort of glioblastoma patients. The model’s predictions, particularly those related to tumor stability, significantly distinguished high-risk from low-risk groups (p=0.017), demonstrating prognostic value comparable to well-established biomarkers such as MGMT methylation status. These findings highlight the potential of LLM-based integration to reduce diagnostic uncertainty and support more personalized treatment strategies in neuro-oncology.

The study, published in the KeAi journal Meta-Radiology, establishes a foundation for future research on multi-source integration in clinical oncology.

“Expanding the approach to include modalities such as MRI imaging and genomic profiles could further enhance predictive power and accelerate progress toward precision cancer care,” adds Ma.

Contact authors:

Zhicheng Jiao, PhD, Brown University Health, zhicheng_jiao@brown.edu

Funder: 

This work was supported by Robert E. Leet and Clara Guthrie Patterson Trust Mentored Research Award.

Conflict of interest:

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

See the article:

Zhuoqi Ma, et al. "Large language model-based multi-source integration pipeline for automated diagnostic classification and zero-shot prognoses for brain tumor." Meta-Radiology 3.2 (2025): 100150. https://doi.org/10.1016/j.metrad.2025.100150