Tulane Researcher Awarded Two NSF Grants to Open Up AI's "Black Box"

Zhengming Ding, an associate professor at Department of Computer Science in Tulane University’s School of Science and Engineering, has received two prestigious National Science Foundation awards to advance interpretable artificial intelligence—research focused on making AI systems more transparent, reliable, and understandable to the people who use them.

The two awards, an EPSCoR Research Fellows grant and a CAREER Award, will support Ding's work over the coming years, as he investigates how AI models make decisions and how to achieve more reliable AI outcome.

“So far, we have treated AI models as a kind of black box.” Ding said. “We don’t know what happens inside. We give the model an input and receive an output, but we don’t necessarily understand why it produces that output. We want to open up the black box, understand the mechanisms behind its decisions, and determine whether those decisions are reliable.”

Ding’s research spans diverse applications, from medical diagnostics and autonomous driving to neuroscience, with a common goal: making AI systems more reliable, trustworthy, and effective.

“AI is fundamentally data driven.” Ding said. “It learns from data, and today we have enormous amounts of it. But some scenarios are rare yet critical, and others may involve inherent conflicts. In these cases, AI may not have enough information to learn how to respond appropriately. We need to identify when AI systems are likely to fail and, more importantly, understand why they fail.”

The EPSCoR Research Fellows award, titled "Interpretable AI-Driven Multi-Modal MRI for Brain Imaging Analysis," began May 1, 2026, and totals $222,497. The project focuses on combining multiple types of brain imaging data to better understand what happens inside the human brain when subjects process different kinds of information.

Ding explained that when researchers collect brain data, subjects are often shown images, audio, or video while their brain activity is recorded. Because different people respond differently based on their personal experiences and preferences, the goal is to understand how those individual differences show up in brain activity.

The project combines multiple types of brain data because no single source can fully capture the complexity of how the brain responds to information. Beyond understanding individual differences in brain activity, the research could eventually help identify differences in how children learn and track changes in cognition over time, potentially providing new insights into brain-related diseases.

“Some children learn very slowly, while others learn very quickly. It’s important to understand their learning trajectories. We can gradually collect data throughout their lives and compare different stages—for example, the beginning and end of a school year. What changes between these two states? We can determine whether they are gradually forgetting something or whether their learning is progressing in a different way.”

Although the project is still in its early stages, Ding said his team has already seen promising results using single-modality data. “We do see different patterns across subjects.” Ding said. “In a sense, we know that different people, when they see the same thing, can produce very different brain signals.”

The work is highly interdisciplinary, drawing on collaboration with neuroscience researchers to validate the AI's findings against established scientific understanding.

“We need scientists from different fields to work together to determine whether our AI discoveries are meaningful and scientifically valid.” Ding said. “That’s why I collaborate with researchers in other disciplines—to bring different types of knowledge together and integrate them with AI.”

Ding also received a CAREER Award, one of NSF's most prestigious honors for early-career faculty, for his project “Toward Interpretable AI: Modeling and Adapting Data-Driven Semantic Prototypes.” The award totals $597,729 and begins July 1, 2026.

While the EPSCoR project applies interpretable AI to a specific application, brain imaging, the CAREER Award is intended to build more foundational, broadly applicable techniques. The project focuses on identifying "prototypes," or concepts, that AI models learn from data and that can be reused and recognized across different scenarios.

“The goal of the prototype is to identify meaningful concepts within the data. These concepts can then be reused in the future, and they should also have meaningful applications for humans.”

The project will also explore how AI can understand patterns in dynamic, changing environments, such as those encountered in autonomous driving. “So far, this is a very new area. There is very little research on this type of problem,” Ding said.

He described the CAREER project as more fundamental and abstract than the brain imaging work, with techniques that could extend to fields well beyond neuroscience, including gene sequence analysis with medical collaborators. “The CAREER project is more fundamental. The techniques can be applied to different domains, including brain decoding, autonomous driving, and gene sequence analysis.” Ding said.

Reflecting on what receiving the CAREER Award means to him, Ding pointed to years of sustained work on this line of research.

“It’s very encouraging because I have been working in this direction for several years.” Ding said. “I think it is very promising to be able to open up the AI black box. Now we have the support to continue this research, which is very exciting for us. We feel greatly encouraged by this support.”

The CAREER Award also includes an education and outreach component, which Ding said is central to the project.

“That’s really the main part of the project—the educational component” Ding said. Ding plans to incorporate the project's concepts into courses he already teaches, including his introduction to AI course, as well as outreach workshops. He said students are often drawn to questions about AI reliability that go beyond standard performance metrics like accuracy.

When asked what he wishes more people understood about interpretable AI, Ding highlighted an important challenge: the way we explain an AI system’s decision may not always reflect how the system actually arrived at that decision. In many fields, researchers rely on language-based explanations to describe AI outputs, even when the underlying data—such as images, brain signals, or other scientific measurements—is not language-based.

He cautioned that this kind of surface-level explanation can be misleading if the underlying patterns behind an AI's language-based output don't actually reflect what happened inside the model.

Ding said his broader ambition is to keep expanding his research across disciplines, including neuroscience and psychology, and to continue integrating domain expertise with data-driven AI models, particularly when working with human data.

“I’m excited to work with researchers from different fields and apply these fundamental techniques to areas such as neuroscience and psychology.” Ding said. “AI models don’t have to be driven by data alone. They can also incorporate domain knowledge, especially when we are working with human data. I’m very open to collaboration.”