Zeyu (Steven) Zhang

PhD Student, Statistics and Data Science, Northwestern University

I am a third-year Ph.D. student in Statistics and Data Science at Northwestern University, working with Prof. Bradly C. Stadie. My research sits at the intersection of statistics, machine learning, and large language models.

I study how to make LLMs more trustworthy and reliable when they are used for prediction and reasoning—especially failure modes such as hallucination, temporal contamination in evaluation, and explanations that look fluent but may not track the model’s actual decisions. I am particularly interested in interpretation of LLMs and in methods that help us audit, constrain, and improve model behavior under realistic deployment conditions.

Research interests

Trustworthy LLMs Reliability Hallucination Interpretation LLM forecasting

News

May 25, 2026	Our new preprint on catching models that “peek” at the future is live on arXiv—with code to reproduce the claim-level leakage audit and TimeSPEC mitigation.
May 13, 2026	TEMPO is out: a training approach that teaches LLMs temporal discipline for trustworthy backtesting. Code released.
Jan 22, 2026	Thrilled that LAMP was selected as a Spotlight at AISTATS 2026—a lightweight way to audit whether a model’s explanations behave like they matter.

Research

Temporal leakage in LLM backtesting

A central challenge in evaluating an LLM’s prediction ability is that testing on the true future is often impractical. The standard alternative is backtesting: ask the model historical questions under a cutoff date, as if it were forecasting from that earlier time. The hard problem is temporal leakage—a special form of data contamination in which post-cutoff knowledge from pretraining seeps into the model’s reasoning, inflating backtest scores and making the evaluation untrustworthy. My work in this line explains and addresses that failure from complementary angles: how leakage breaks evaluation validity, how to block it at inference time, and how to train models for temporal discipline.

Selected works in this line

Measure & mitigate at inference TimeSPEC

All Leaks Count, Some Count More

Backtesting assumes models use only pre-cutoff evidence, but pretrained LLMs routinely leak later knowledge into their rationales. We introduce Shapley-DCLR to measure how much decision-critical reasoning is contaminated at the claim level, and TimeSPEC, an inference-time supervisor that regenerates predictions from temporally filtered evidence without retraining.

Preprint · Under review at EMNLP 2026

Abs arXiv PDF Code

Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level evaluation framework that decomposes prediction rationales into atomic claims and applies Shapley values to quantify each claim’s decision impact, yielding Shapley-DCLR (Shapley-weighted Decision-Critical Leakage Rate) – an interpretable metric measuring what fraction of decision-driving reasoning is contaminated. We further propose TimeSPEC (Time-Supervised Prediction with Extracted Claims), an inference-time architecture that interleaves temporally-filtered retrieval with claim-level supervision, producing predictions grounded entirely in pre-cutoff evidence. Across three LLMs, the ablation experiments confirm retrieval and supervision are jointly necessary; and a three-task probe further illstrates that the performance cost of temporal enforcement scales with each task’s reliance on post-cutoff information.

Mitigate via training TEMPO

Temporal Enforcement via Mode-Separated Policy Optimization

Temporal leakage lets LLMs recall post-cutoff outcomes during backtesting, so reported accuracy is often invalid. TEMPO trains temporal discipline with a two-mode RL objective that first drives leaked claims to zero, then optimizes prediction quality once reasoning stays pre-cutoff.

Preprint · Under review at NeurIPS 2026

Abs arXiv PDF Code

Backtesting large language models on historical events requires reasoning exclusively from information available before a specified cutoff date. Yet models routinely leak post-cutoff knowledge from pre-training into their reasoning, inflating apparent accuracy and undermining evaluation validity. Prompt-based constraints fail when suppressed content is causally related to the prediction, and knowledge unlearning cannot address this problem because temporal compliance is instance-specific: the same fact may be legitimate evidence for one cutoff date and a violation for another. Rather than erasing knowledge, the model must learn temporal discipline: selecting evidence conditioned on each instance’s cutoff date. We propose TEMPO (Temporal Enforcement via Mode-separated Policy Optimization), which trains this discipline via two contributions: (1) a two-mode reward where a leakage mode drives post-cutoff claims to zero as a hard prerequisite before a performance mode optimizes task performance; and (2) a GRPO-based training pipeline that enables the model to discover temporally valid reasoning strategies. We prove that training monotonically decreases leakage, converges to the leak-free optimum, and improves task performance once compliance is achieved. On three prediction tasks and two models, TEMPO reduces leakage from 2–13% to 0.6–3.7% across all conditions, with task performance improving 6–13% where strong pre-cutoff signals exist and maintained where the prediction task is inherently difficult from valid information alone.

Interpretation: do explanations match decisions?

LLMs often give fluent explanations for their predictions, but fluency is not faithfulness. The hard question is whether the factors a model cites actually behave like they drive the decision—and how to check that without gradients or internal access. That makes interpretation both practically needed for trust and scientifically interesting as a window into model behavior.

Selected works in this line

Local decision surfaces LAMP

Extracting Locally Linear Decision Surfaces from LLM World Models

Fluent LLM explanations need not track what actually drives the prediction. LAMP treats the model’s own stated factors as coordinates and fits a local linear decision surface—without gradients or internal access—to test whether predictions move consistently when those factors change.

Spotlight paper in AISTATS 2026

Abs arXiv PDF

We introduce LAMP (Local Attribution Mapping Probe), a method that shines light onto a black-box language model’s decision surface and studies how reliably a model maps its stated reasons to its reported predictions by approximating a decision surface. LAMP treats the model’s own self-reported explanations as a coordinate system and fits a locally linear surrogate that links those weights to the model’s output. By doing so, it reveals how much the stated factors steer the model’s decisions. We apply LAMP to three tasks: sentiment analysis, controversial-topic detection, and safety-prompt auditing. Across these tasks, LAMP reveals that many language models’ locally approximated linear decision landscapes overall agree with human judgments on explanation quality and, on a clinical case-file data set, align with expert assessments. Since LAMP operates without requiring access to model gradients, logits, or internal activations, it serves as a practical and lightweight framework for auditing proprietary language models, and enabling assessment of whether a model appears to behave consistently with the explanations it provides.