Assessing the suitability of generative AI for literature retrieval

The incorporation of generative artificial intelligence (AI) and large language models like ChatGPT into research methodologies has elicited both enthusiasm and trepidation. Although the potential for simplifying tasks and boosting efficiency is appealing, the risk of inaccuracies and “hallucinations” presents a substantial hurdle.

In a new study published in Pharmacoeconomics and Policy, a team from Tulane University Celia Scott Weatherhead School of Public Health and Tropical Medicine in New Orleans assessed ChatGPT’s effectiveness in conducting literature reviews.

Employing established criteria from systematic reviews on chronic illnesses, the authors instructed ChatGPT to locate pertinent articles. Notably, the outcomes were troubling. Despite generating a list of article titles, a substantial fraction (42%) was either fictitious or misattributed, underscoring the problem of “hallucinations” in AI-generated content. Only 18% of the citations corresponded to the actual articles included in systematic reviews.

“ChatGPT can be a valuable asset for researchers, but it’s not without flaws,” asserts Debra Winberg, lead author of the study. “Our results highlight the constraints of current AI algorithms, and the necessity for researchers to critically assess and validate information produced by AI tools.”

“The swift advancement of AI tools is promising, but we must ensure their responsible and effective use,” adds co-author Dennis Xuan. “Currently, generated information is heavily influenced by the phrasing of the question, the specified criteria and publicly available resources. Hence, healthcare decision and policy makers cannot yet rely on pure generative AI output without knowing whether humans were involved in the entire research process.”

In summary, there appears to exist an ability ceiling above which the current ChatGPT algorithms cannot reach.

Contact author: Dennis Xuan, Tulane University, New Orleans, USA

dxuan@tulane.edu

Conflict of interest: The authors declare that there are no conflicts of interest.

See the article: https://doi.org/10.1016/j.pharp.2025.03.003