For internal use only
UI Test Automation with
SpecFlow
Presented by Eugeny Govako
For internal use only
Overview
•
Gherkin and test automation
•
NUnit based automation approach
•
SpecFlow automation approach
•
SpecFlow automation evolution
•
Q&A
For internal use only
Gherkin and test automation
•
Test Automation is implementing
specifications step by step using
some programming language
•
Gherkin is the language for
executable specifications
•
Gherkin itself is a kind of a
programming language with well
defined syntax
•
Originally invented to use with
Cucumber framework
For internal use only
Prerequisites
Gherkin
•
Specifications (written in Gherkin preferably)
•
Test API
•
Driver providing access to Application with ability to find required
control
•
Wrappers for controls
•
Application under test
Test
TestAPI
Application
For internal use only
Non-SpecFlow automation
•
NUnit tests implementation based on Gherkin lines
For internal use only
Non-SpecFlow automation summary
Test
TestAPI
Application
Gherkin
•
No direct link between specification (Gherkin) and test code
•
Test code uses Test API layer that provides access to
controls, allowing application interaction
For internal use only
SpecFlow automation
Specification linked to test
Test development is very fast
Automation code base grows even more fast
Test Support trouble
For internal use only
SpecFlow automation: basics
•
Using parameters to reduce code amount
For internal use only
SpecFlow automation: basics
•
Tables can be used row-by-row
For internal use only
SpecFlow automation summary
Gherkin
SpecFlow
engine
Bindings
Test API
Application
•
Tests code is directly connected to Gherkin via SpecFlow
runner
•
Test API layer still used for providing controls
•
Binding method arguments reduce code amount
Parameters
For internal use only
Gherkin requirements for automation
•
Reuse steps
•
Use parameterization where possible
Any Gherkin line can be automated. However,
there’s always tradeoff between effort needed to
create scenario and effort needed to automate.
For internal use only
SpecFlow automation: advanced features
•
Using step argument transformations
For internal use only
•
Tables can be used in transformations
•
Tables can be converted to parameter maps
SpecFlow automation: advanced features
For internal use only
•
Trouble?
•
Parameterization leads to huge maps of controls
•
Conflicting Regex for same actions in different contexts
•
Code base still grows fast
•
How?
•
Execution Context – specify where to look for control
•
Use direct transformations to automation wrappers
•
Register controls with friendly names in a context-specific resolver
•
Silver Bullet?
•
Makes it possible to have bindings provided from the box, along
with control wrappers
SpecFlow automation: generic approach
For internal use only
Example: working with buttons
•
IButton wrapper
•
Provides access to Visibility, enabled state, text, click functionality
•
Implementation comes from automation framework
•
Visual Resolver:
•
Allows registering button by friendly name
•
Allows resolving button in an active context
•
Automatically transforms control requested by friendly to DSL
interface used inside binding method
•
StepArgumentTransformation returning IButton
•
Using Visual Resolver, convert friendly name (string) into IButton
wrapper instance
•
Button Bindings extending ControlBindingBase<IButton>
•
Contain bindings that use string -> IButton conversion
SpecFlow automation: control-based
For internal use only
Visual Resolver example
SpecFlow automation: control-based
For internal use only
Control binding class example
SpecFlow automation: control-based
For internal use only
•
Uses SpecFlow native step interpreter
•
Uses Regexp for steps and parameters mapping
•
Uses Visual Resolver to convert names to DSL
Control-based approach: summary
Reduced number of steps
Development costs are low
Diagnostics is not good
Similar steps still exist for
different control types
For internal use only
Control based example:
SpecFlow automation: behavior driven
4 binding
methods
Scenario
Block
Control
friendly name
Control
type
What to
check
Behavior driven example
WHAT
HOW
Control wrapper from
Visual Resolver
Behavior control
should implement
For internal use only
Behavior-driven approach
Behavior
Interface
Property
shown / not shown
ICanBeVisible
IsVisible
enabled / disabled
ICanBeEnabled
IsEnabled
has “some” text
IHasVisibleText
Text
•
Works with all controls supporting behaviors
designated by atomic interfaces
•
New behaviors can be introduced easily
For internal use only
Behavior driven approach: conclusion
Automation bindings number significantly reduced
Focus on behavior, not control (closer to business)
Still risks of conflicts hard to resolve via regexp
For internal use only
Romashka Driver
•
Give up Regexp!
•
Use smart step parser and grammar rules
•
Find controls and analyze/apply behaviors
No Regexp conflicts
New behaviors added easily
Support new controls by
applying behavior interfaces
to DSL
Diagnostics is even worse
Hard to understand and
contribute
Crush a fly upon a wheel
feeling
For internal use only
Specflow automation in Autobahn FX UI
•
100% P1 / 80% all automation coverage (of possible to
automate) in 5 projects using SpecFlow
•
More than 23000 test cases automated
•
Reasonable test support costs
For internal use only
Q&A
Исполнимые спецификации в тестировании UI