Simulation as a Dataset Engine
Generating high-fidelity traces for agent evaluation.
Type
Research Direction
Author
Brandon Butera
Publication
Published
Evidence
Prototype-Informed
Published
November 3, 2025
Systems
boundary
Evaluation is the bottleneck of AI safety. We cannot wait for agents to fail in production to learn how to secure them.
### Synthetic Traces
Boundary generates synthetic traces by running agents against a set of scenario specifications. These scenarios include edge cases like network timeouts, conflicting instructions, and adversarial tool outputs.
### The Replay Factor
Every trace generated is replayable. This allows us to re-run the same scenario with a modified governance policy to see if the vulnerability is closed. We believe this 'simulation-first' approach is the only way to build truly robust systems for real-world operational settings.