Software quality assurance days International Conference of Software Quality Assurance sqadays.com Minsk. May 29–30 2015 Kovalenko Andrey Lohika. Odesa, Ukraine Way to performance testing. How to pass it from automation.
Performance testing for Automation QA – why and how? Andrey Kovalenko 6 last years in IT - again 3 years of developing and implementing distributed protected networks 3 last years - performance engineer kovalenko.andrey.odessa About
Performance testing for Automation QA – why and how? What is performance? Load testing - to verify application behavior under normal and peak load conditions. Stress testing - To determine or validate an application’s behavior when it is pushed beyond normal or peak load conditions. Capacity testing - To determine how many users and/or transactions a given system will support and still meet performance goals.
Performance testing for Automation QA – why and how? Why do we need performance?
Performance testing for Automation QA – why and how? Performance VS Functional testing Based non-functional aspects of the software system Non functional testing should be performed after functional testing Almost impossible to run manually Describes HOW GOOD the product works Based on functional specification Functional testing is executed first Easy to do manual testing Describes WHAT the product does Functional manual Functional manual Performance Performance Functional Automation Functional Automation Based on functional specification Automation should be performed after manual functional testing Impossible to run manually Describes WHAT the product does
Performance testing for Automation QA – why and how? Software Testing Life Cycle
Performance testing for Automation QA – why and how? STLC for performance testing 1. Identify requirements 1. Features related to performance – with devs 2. Modules related to performance – with devs and FA/SA 3. Analyzing application’s tiers 2. Acceptance criteria (Performance KPIs): 1. Main TRT values – from your FA and public sources 2. Main system resources threshold – from your experience 3. Strategy planning and design: 1. Customer scenario use – from your FA 2. 2. Transaction defining Transaction defining – from your experience and FA 3. Scripts writing 4. 4. Production-like environment establishing Production-like environment establishing can be a problem can be a problem 5. 5. Monitors for resources – native, from specific tools Monitors for resources – native, from specific tools
Performance testing for Automation QA – why and how? STLC for performance testing 4. Execute & Analyze 1. Run performance scenario 2. Collect results and resources’ consumption data 3. Analyze results, comparing with KPIs 4. 4. Analyze source code to find possible bottlenecks Analyze source code to find possible bottlenecks 5. 5. Analyze logs, heap GC, dumps Analyze logs, heap GC, dumps 6. 6. Create recommendations for application tuning – for application server, Create recommendations for application tuning – for application server, code, hardware, software code, hardware, software 7. Prepare reports for stakeholder
Performance testing for Automation QA – why and how? Performance VS Automation Multi-user scenario Interested in response time Should be ran on production-like env Resources must be monitored, application should be profiled Single user Interested in response data Can be ran on any hardware No needs to monitor resources Automation Automation Performance Performance