AliExpress Performance Test System
--
All chains and regular stress test
About me
•
Name: JUNYUAN HU (Joy)
•
Position: Test-Dev Specialist in AliExpress
•
Responsibility for: Quality System
Construction in AliExpress Moscow office,
All chains stress test in AliExpress
2018/12/10
2
•
The Globalization process is speed up
–
We served buyers from more than 220 countries and regions
–
We got 100 million buyers in April 2017
2018/12/10
3
Challenges of biz & tech in AliExpress
•
Traffic is continuous growing up quickly
2018/12/10
4
Challenges of biz & tech in AliExpress
What business type should we serve
2018/12/10
5
Business
needs in
AliExpress
Big Promotion
Daily operating
Quickly
Disaster
recover
Drainage
Local site
Oversea IDC
Notice Push via
App
Changes in projects
Flexible
capacity
expansion
Anniversary Promotion
Brand shopping week
11.11
New Year Promotions
Daily promotion activities
2018/12/10
6
Big Promotions in AliExpress
Important terms
•
GMV: Gross Merchandise Volume
•
QPS: Query per second
•
DAU: One-Day active users
•
SPM: Super position model
•
IDC: Internet Data Center
•
Perf: abbreviation from of performance
•
Shadow table: a copy of data base with special prefix, has
the same table structure with formal table
•
Pre-prod env: an separate machine in the cluster for test
before release branches to production environment
2018/12/10
7
What service should we provide
for performance test
2018/12/10
8
Performance
monitoring
in local site
Regular
performance test
in every week
Performance
Benchmarking data
accumulation
Monitor overview
Performance test
in Project
Data Prepare
Services in
performance
test
What abilities should we have
in performance test
2018/12/10
9
Monitoring
aggregation
Real-time water level
Utilization rate of
servers
PC&M site perf
monitoring
App perf monitoring
Monitoring
system
All chains perf test
Scene modeling
Traffic forecasting
model
Regular performance
test management
Links management
Core applications
management
Links analyst
Management
system
Benchmarking in pre
-prod
Benchmarking in
trunk
Performance
Benchmarking
Data prepare on
cloud
Data
Implemented
Online
performance test
Performance test
in pre-prod env
Performance test
in dev env
One Key
performance test
Performance
test execute
Developing
Apps *
Machines
Apps *
Indicators
Apps *
QPS
Entrance
PV
Push
Advertising
Strategy
Other
DAU
Detail
Search
Shopping cart
Place order
feedback
QPS
(Shopping cart detail)
QPS
(Add to cart)
QPS
(Confirm order)
QPS
(Place order)
Shopping cart detail QPS
CPU
Add 2 cart QPS
CPU
Confirm order QPS
CPU
Place order
QPS
CPU
GMV
Homepage
Evaluation Criteria
Performance test module
Capacity forecast
Biz
Target
Performance
Test Data
Cloud
Monitor
Data of system performance
Monitor data
Capacity of
Apps
Water
level
Data & Script Prepare
2018/12/10
11
Data purification
Data of read chains
Access Log
Data Split
Scene match
Data customize
mathematical
operation
Data of write chains
Shadow table data
polymerization
Customized background data
Traffic
analyst
Data of performance module
RBI data
Time stamp
selection
Biz decision
ROI analyst
Invoke
chains
Data & Scenes
Biz target
Business logic and scenes management
2018/12/10
12
w/r data tag
Data association of
scenes
Chains & scenes
QPS of
scenes
Peak value of read module
Peak
value of write module
Peak value of search module
Others modules, etc.,
Management System
2018/12/10
13
Capacity manage
Compare with
benchmarking
Capacity forecast
Capacity calculate
Capacity
forecast
Chains & Scene manage
Test Requirement
management
Module manage
Dispatch Script
Benchmarking
management
Management
Execute and
calculate at
background
The Pipeline of All chains performance test
2018/12/10
14
Big promo KO
Stress test
target confirm
Prepare test
data
Indicators & system
performance
Capacity
Stress test
Biz target
transfer to
technical
grantee target
Output
performance
test
target/pressure
module
Data generation
rules
Generate data
automatically
Data
proofreading
Analyst of
Data & stress
System
capacity
needs
Performance
verify
Terminal target
:
Regular performance test + Capacity forecast=>Capacity of big promotion
!!
!
Data
Biz
Target
Pages
amount
Stress test
module
Different
QPS
System
performance
Different QPS
proportion
System
capacity
QPS
distributed in
Apps
Data
Data
Data
Data
2018/5/27
15
Regular Performance test
16
AE
性能
测试体系架构
Local site perf monitoring: AGP
实
现
自
动
化
17
Performance benchmarking
实
现
自
动
化
18
AE
性能
测试体系架构
Quick Performance test platform
实
现
自
动
化
OP Quick
Performance
test
Online(CN/US/RU)
/Offline
HTTP(s)
Performance test task
type
HSF
Mixed HSF
Combination scenes
Pre-Prod Benchmarking
(
Deployed
)
Benchmarking
Trunk Benchmarking
(
Timing
)
Offline monitor
(
TSAR
,
JVM Data
Collection
)
Performance test monitor
Online Monitor
(
Alimonitor data
Collection
)
Execute
Result
Task
tags
…
Alarm
notice
Data
prepare
Mock
login
Customized
2018/12/10
19
Thank you!
Email
DingTalk
Facebook
The all links performance test in AliExpress