Portrait of Isabella Beltran Shapland

Mechanistic ML Researcher

Isabella Beltran Shapland

I work on RL alignment, visual reasoning, and model reliability. Lately that means teaching vision-language models to keep looking at the image.

Open to research roles in AI safety and model behavior

about

I study how to make imperfect signals from inside a model safe to use in RL. My current work puts visual-attention probes into GRPO rewards for vision-language models, with constraints that let the probe break ties but never override correctness.

I did my B.S. in Computer Science at UC Santa Cruz (2022–2026), where I pitched and first-authored a GRPO research paper in Dr. Yuyin Zhou's lab. Before that I worked on explainable ARC-AGI reasoning at TrainLoop (YC W25) and on on-device LLM tooling at Intel. Lately I've been spending my own time on interpretability for safety: faithfulness, reward hacking, and eval awareness.

research interests

news

research

  1. Break the Loop with Good Attention: Visual-Attention-Based Probes in GRPO Rewards to Mitigate Repetition Collapse

    Isabella Beltran Shapland, Hardy Chen, Cihang Xie, Yuyin Zhou

    Committed to EACL 2027 · ACL Rolling Review, Aug 2026

    Reasoning VLMs trained with GRPO can fall into repetition collapse when their attention drifts off the image. A frozen probe on visual attention (0.796 AUROC) is used only to break reward ties, never to override task correctness. It also recovers a learning signal from fully tied groups, where vanilla GRPO gives no gradient at all.

    A model answers the same question 8 times. Training then pushes each answer right (do more of this) or left (do less). Bar length is how hard.

    ✓ right✗ wrongright answers always stay above wrong onesClick After to see what changes
    Reads both graphs, then commits
    Clean steps, but assumes the wrong height
    Right answer after ~2,700 tokens of “Wait…”
    Checks the setup once, then answers
    Misreads the graph and loops
    Grounded and concise
    Repeats itself, then guesses
    Long but steady derivation

    Plain GRPO only knows right or wrong. All five right answers get the same push, so the one that looped for 2,700 tokens is rewarded exactly as much as the clean ones.

    The "push" is GRPO's advantage. Example answers and probe scores are illustrative, modeled on the paper's case studies; the pushes are computed with the paper's formulas (Eq. 1, 4, 5; α = 0.5).

    View as table
    AnswerResultLooks at imagePush beforePush after
    Reads both graphs, then commits✓ right0.75+0.72+0.97
    Clean steps, but assumes the wrong height✗ wrong0.84−1.21−0.72
    Right answer after ~2,700 tokens of “Wait…”✓ right0.36+0.72+0.24
    Checks the setup once, then answers✓ right0.62+0.72+0.72
    Misreads the graph and loops✗ wrong0.38−1.21−1.21
    Grounded and concise✓ right0.81+0.72+1.21
    Repeats itself, then guesses✗ wrong0.21−1.21−1.69
    Long but steady derivation✓ right0.55+0.72+0.48
    • +3.4 in-domain accuracy
    • +6.3 on MathVision
    • ~⅓ less repetition
    • Qwen3-VL-8B, full fine-tune

    [pdf]

  2. ARC-AGI v2: Explainable Reasoning Traces for GPT-5-Nano; The Execution Bottleneck

    Isabella Beltran Shapland, Jackson Stokes

    TrainLoop research report, 2025

    GPT-5-nano solves ARC-AGI v2 puzzles by writing illustrated instruction booklets made of IF-THEN steps. Its reasoning was fully correct on 45.5% of 33 human-annotated puzzles, compared with 0.9% raw benchmark accuracy. Execution then failed in 93.8% of those cases, so the bottleneck is visual execution, not reasoning.

    • 45.5% correct reasoning
    • 93.8% execution failure
    • failure-mode taxonomy

    [pdf]

experience

  1. UC Santa Cruz

    Jan 2026 – Present

    First-Author ML Researcher · Dr. Yuyin Zhou's lab · Santa Cruz, CA

    • Pitched the project in office hours, then built all of it: the Verl training setup, RL configs, data ingestion, controls, and multi-arm ablations on 8×A100 and 2×H200.
    • Designed the tie-only injection method and showed why simply adding the probe to the reward degrades response quality.

  2. TrainLoop (YC W25)

    Sept 2025 – Dec 2025

    ML Research Intern · San Francisco, CA

    • Built a 6-stage pipeline that turns multimodal puzzle-solving into auditable reasoning booklets with a crop-transform-uncrop image loop.
    • Designed the annotation schema and failure-mode taxonomy that made reasoning and execution measurable separately.

    [paper]

  3. Intel

    May 2025 – Sept 2025

    ML Software Engineering Intern · Santa Clara, CA

    • Built a Haystack RAG pipeline (LLaMA-3.2) that pulls CPU anomalies from large JSON system logs, plus a deterministic Python validator to catch hallucinations.
    • Wrote Intel's first internal benchmarks for local model hosting on integrated GPUs (Ollama, SYCL).

    [BLDT v3.0 write-up]

  4. ACM at UCSC

    Sept 2023 – June 2024

    ML Researcher · Santa Cruz, CA

    • Tested BERT against gradient-based Universal Trigger Token attacks and developed a new robustness measure from the results.
    • Ran the attack simulations in parallel across GPU nodes with Kubernetes and PyTorch.

projects

talks

misc

work in progress

An independent interpretability project on AI safety and model behavior. It's ongoing, so I describe the question and method here, not results.