For internal use only UI Test Automation with SpecFlow Presented by Eugeny Govako
For internal use only Overview • Gherkin and test automation • NUnit based automation approach • SpecFlow automation approach • SpecFlow automation evolution • Q&A
For internal use only Gherkin and test automation • Test Automation is implementing specifications step by step using some programming language • Gherkin is the language for executable specifications • Gherkin itself is a kind of a programming language with well defined syntax • Originally invented to use with Cucumber framework
For internal use only Prerequisites Gherkin • Specifications (written in Gherkin preferably) • Test API • Driver providing access to Application with ability to find required control • Wrappers for controls • Application under test Test TestAPI Application
For internal use only Non-SpecFlow automation • NUnit tests implementation based on Gherkin lines
For internal use only Non-SpecFlow automation summary Test TestAPI Application Gherkin • No direct link between specification (Gherkin) and test code • Test code uses Test API layer that provides access to controls, allowing application interaction
For internal use only SpecFlow automation Specification linked to test Test development is very fast Automation code base grows even more fast Test Support trouble
For internal use only SpecFlow automation: basics • Using parameters to reduce code amount
For internal use only SpecFlow automation: basics • Tables can be used row-by-row
For internal use only SpecFlow automation summary Gherkin SpecFlow engine Bindings Test API Application • Tests code is directly connected to Gherkin via SpecFlow runner • Test API layer still used for providing controls • Binding method arguments reduce code amount Parameters
For internal use only Gherkin requirements for automation • Reuse steps • Use parameterization where possible Any Gherkin line can be automated. However, there’s always tradeoff between effort needed to create scenario and effort needed to automate.
For internal use only SpecFlow automation: advanced features • Using step argument transformations
For internal use only • Tables can be used in transformations • Tables can be converted to parameter maps SpecFlow automation: advanced features
For internal use only • Trouble? • Parameterization leads to huge maps of controls • Conflicting Regex for same actions in different contexts • Code base still grows fast • How? • Execution Context – specify where to look for control • Use direct transformations to automation wrappers • Register controls with friendly names in a context-specific resolver • Silver Bullet? • Makes it possible to have bindings provided from the box, along with control wrappers SpecFlow automation: generic approach
For internal use only Example: working with buttons • IButton wrapper • Provides access to Visibility, enabled state, text, click functionality • Implementation comes from automation framework • Visual Resolver: • Allows registering button by friendly name • Allows resolving button in an active context • Automatically transforms control requested by friendly to DSL interface used inside binding method • StepArgumentTransformation returning IButton • Using Visual Resolver, convert friendly name (string) into IButton wrapper instance • Button Bindings extending ControlBindingBase<IButton> • Contain bindings that use string -> IButton conversion SpecFlow automation: control-based
For internal use only Visual Resolver example SpecFlow automation: control-based
For internal use only Control binding class example SpecFlow automation: control-based
For internal use only • Uses SpecFlow native step interpreter • Uses Regexp for steps and parameters mapping • Uses Visual Resolver to convert names to DSL Control-based approach: summary Reduced number of steps Development costs are low Diagnostics is not good Similar steps still exist for different control types
For internal use only Control based example: SpecFlow automation: behavior driven 4 binding methods Scenario Block Control friendly name Control type What to check Behavior driven example WHAT HOW Control wrapper from Visual Resolver Behavior control should implement
For internal use only Behavior-driven approach Behavior Interface Property shown / not shown ICanBeVisible IsVisible enabled / disabled ICanBeEnabled IsEnabled has “some” text IHasVisibleText Text • Works with all controls supporting behaviors designated by atomic interfaces • New behaviors can be introduced easily
For internal use only Behavior driven approach: conclusion Automation bindings number significantly reduced Focus on behavior, not control (closer to business) Still risks of conflicts hard to resolve via regexp
For internal use only Romashka Driver • Give up Regexp! • Use smart step parser and grammar rules • Find controls and analyze/apply behaviors No Regexp conflicts New behaviors added easily Support new controls by applying behavior interfaces to DSL Diagnostics is even worse Hard to understand and contribute Crush a fly upon a wheel feeling
For internal use only Specflow automation in Autobahn FX UI • 100% P1 / 80% all automation coverage (of possible to automate) in 5 projects using SpecFlow • More than 23000 test cases automated • Reasonable test support costs
For internal use only Q&A
Исполнимые спецификации в тестировании UI