Streaming Failure Forecasting

Robot failure monitors that keep learning after deployment

A robot policy that worked in the lab meets dimmer light, new objects, a bumped camera and worn grippers in the field. We study failure monitors that keep learning from how each episode ended, so they still raise the alarm when conditions drift.

00.510 s5 s10 sfrozen thresholdSFF thresholdSFFfrozenSFF alarm, 3.5 s
A logged simulation episode with gripper fingers that wore out unseen by the camera. Our monitor (SFF) has learned from earlier episodes and raises the alarm 3.5 s in; the same monitor kept frozen never does, and the episode fails at its time limit. Step through this episode.

Research

  1. From simulation to the SO-101

    Our real-arm rig is built and recording: a low-cost SO-101, three cameras at 30 fps and AprilTags that label every episode automatically.

  2. Failure monitors that keep learning after deployment

    A monitor that keeps learning from every episode outcome ranked failures better than a frozen one on 4 of 5 simulation test suites, including the pre-registered primary test.

  3. Early experiments: a language model driving two arms

    With no demonstrations and no training, Claude Opus 5.5 planned every move of two SO-101 arms and handed an eraser from one arm to the other.

  4. A better alarm is not always a better forecast

    How we keep our evaluation honest: ranking, false alarms and warning time, each measured on its own, so a timer can never pass as a forecaster.