A Journey to the East for Quality

When an AI agent takes on tests, it appears confident and almost always makes big mistakes. Run it through your code, and it writes a neat “test” that doesn't check anything. It marks red as green, passes off a bug as a feature, and even accepts an incorrect result as correct.

We experienced this ourselves: we transformed a set of disparate scripts into a system of agents with a clear division of roles. Along the way, we had to answer nine awkward questions—exactly the same ones that are the topics of this talk. What is considered a bug, and what is a feature? How do we deal with old services without documentation? What happens when a model is changed?

This talk is a map of our experience, not a textbook: an analysis of real problems and specific solutions.