ChatGPT matched epileptologists on seizure localization in 100-question survey

ChatGPT matched or outperformed five board-certified epileptologists at predicting epileptogenic zone locations in the brain's most commonly affected regions, according to a study published in the Journal of Medical Internet Research on May 12, 2025. The comparison covered 100 questions on seizure semiology interpretation, a core step in presurgical epilepsy workup.

The stakes are large. Epilepsy affects more than 70 million people worldwide, and about 3.4 million of them live in the United States. Roughly one third of cases do not respond to medication. For those patients, resective surgery that removes the epileptogenic zone can reduce or eliminate seizures — yet the current success rate sits at only 50 to 60 percent, partly because the zone is inaccurately identified.

Feng Liu, Assistant Professor in the Department of Systems and Enterprises at the Schaefer School of Engineering and Science, Stevens Institute of Technology, and his collaborators designed the study to test whether a large language model could interpret semiology text as reliably as specialists. The findings appear in the Journal of Medical Internet Research, volume 27, article e69173 (DOI: 10.2196/69173).

How did the researchers run the comparison?

The team surveyed five board-certified epileptologists through an online questionnaire of 100 questions. Each question gave a description of seizure semiology and asked for the likely location of the epileptogenic zone. The team then gave ChatGPT the same task and scored both against the reference answers.

The clinical background matters here. Patients being considered for resection typically undergo MRI, scalp electroencephalography and intracranial EEG. Epileptologists read those data together with the observed semiology — the symptoms and behaviors during seizures — to predict where seizures originate.

In routine presurgical workup, the model could serve as a second reader for semiology text before intracranial implantation decisions.

Where did the model win and where did it lose?

ChatGPT matched or beat the epileptologists for regions where epileptogenic zones commonly occur, including the frontal lobe and the temporal lobe. The specialists retained the advantage in rarely affected regions, such as the insula and the cingulate cortex.

The takeaway: the model handles the high-frequency localization questions well but should not be trusted alone for rare anatomical targets.

Why does terminology cause surgical problems?

Semiology descriptions vary across epilepsy centers, and that variance complicates surgical planning. "Different epilepsy centers may use different terms describing the same seizure semiology," Liu said. He cited the example of "asymmetric posturing" and "asymmetric tonic activity" — two labels for the same posture, one limb extended while the other is flexed.

"Large language models such as ChatGPT could be valuable tools for analyzing complex textual information, helping interpret seizure semiology descriptions and assist in accurately localizing the epileptogenic zones," Liu said. LLMs trained on vast public corpora can map divergent terms onto a single concept, which is exactly the inconsistency problem that semiology reporting poses.

What is EpiSemoLLM?

To push performance further, the team built EpiSemoLLM, described as the first LLM specialized for interpreting seizure semiology. It runs on a GPU server at Stevens. The platform is intended as a decision-support assistant for neurosurgeons and epileptologists during the presurgical workup phase, not as an autonomous diagnostic tool.

The study tested ChatGPT and a fine-tuned model on textual semiology interpretation; it did not involve direct analysis of EEG waveforms or intracranial recordings. Both the device and the workflow remain investigational, with no FDA clearance claimed for any component.

"Our results demonstrate that LLM and fine-tuned LLM might serve as a valuable tool to assist in the preoperative assessment for epilepsy surgery," Liu said. "The best results would be for the humans and AI to work together."

The next step for the Stevens group is validating EpiSemoLLM prospectively in presurgical cases, particularly in the rare-region localization where the general model currently lags behind specialists.