Designing pass/fail criterias for performance testing and analysis

When you start the performance testing activity for you project one of the first questions you face is - "What results can be classified as success or failed?". To make these decisions you need so called "budgets" - upper nd lower bounds for metrics which characterize system's performance. But ususally perfromance testing starts not at the beggining of the product's development but in active development phase and this fact brings additional problems to budgets choose. Performance control and improvement strongly depends on proper budgets design and their integration to project's lifecycle.

In this presentation different approaches to budgets design with their procs and cons will be covered. Also I'll demonstrate the system of multi-level budgets - so called "performance indicators" and how this system is applied to "World Of Warships" game development process.