Workloads
adobereader
The Adobe Reader workflow carries out the following typical productivity tasks.
Test description:
Open a local file on the device
- Gestures test:
2.1. Swipe down across the central 50% of the screen in 200 x 5ms steps 2.2. Swipe up across the central 50% of the screen in 200 x 5ms steps 2.3. Swipe right from the edge of the screen in 50 x 5ms steps 2.4. Swipe left from the edge of the screen in 50 x 5ms steps 2.5. Pinch out 50% in 100 x 5ms steps 2.6. Pinch In 50% in 100 x 5ms steps
- Search test:
Search
document_name
for each string in thesearch_string_list
Close the document
Known working APK version: 19.7.1.10709
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- document_name:
type:
'str'
The document name to use for the Gesture and Search test.
default:
'uxperf_test_doc.pdf'
- search_string_list:
type:
'list_of_strs'
For each string in the list, a document search is performed using the string as the search term. At least one must be provided.
constraint:
len(value) > 0
default:
['The quick brown fox jumps over the lazy dog', 'TEST_SEARCH_STRING']
aitutu
Executes Aitutu Image Speed/Accuracy and Object Speed/Accuracy tests
The Aitutu workflow carries out the following tasks. 1. Open Aitutu application 2. Download the resources for the test 3. Execute the tests
Known working APK version: 1.0.3
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
androbench
Executes storage performance benchmarks
The Androbench workflow carries out the following typical productivity tasks. 1. Open Androbench application 2. Execute all memory benchmarks
Known working APK version: 5.0.1
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
angrybirds_rio
Angry Birds Rio game.
The sequel to the very popular Android 2D game.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.
antutu
Executes Antutu 3D, UX, CPU and Memory tests
Test description: 1. Open Antutu application 2. Execute Antutu benchmark
Known working APK version: 8.0.4
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
Specify the version of Antutu to be run. If not specified, the latest available version will be used.
allowed values:
'7.0.4'
,'7.2.0'
,'8.0.4'
,'8.1.9'
,'9.1.6'
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
apache
Load-test an apache installation by issueing parallel requests with ab.
Run ab, the Apache benchmark on the host, directed at the target as the server.
Note
It is assumed that Apache is already running on target.
Note
Current implmentation only supports a very basic use of the benchmark.
aliases
ab
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- port:
type:
'integer'
Port on which Apache is running.
default:
80
- path:
type:
'str'
Path to request.
default:
'/'
- parallel_requests:
type:
'integer'
The number of parallel requests at a time.
default:
350
- total_requests:
type:
'integer'
The total number of parallel requests.
default:
100000
applaunch
This workload launches and measures the launch time of applications for supporting workloads.
Currently supported workloads are the ones that implement ApplaunchInterface
. For any
workload to support this workload, it should implement the ApplaunchInterface
.
The corresponding java file of the workload associated with the application being measured
is executed during the run. The application that needs to be
measured is passed as a parameter workload_name
. The parameters required for that workload
have to be passed as a dictionary which is captured by the parameter workload_params
.
This information can be obtained by inspecting the workload details of the specific workload.
The workload allows to run multiple iterations of an application launch in two modes:
Launch from background
Launch from long-idle
These modes are captured as a parameter applaunch_type.
launch_from_background
Launches an application after the application is sent to background by pressing Home button.
launch_from_long-idle
Launches an application after killing an application process and clearing all the caches.
Test Description:
During the initialization and setup, the application being launched is launched for the first time. The jar file of the workload of the application is moved to device at the location
workdir
which further implements the methods needed to measure the application launch time.- Run phase calls the UiAutomator of the applaunch which runs in two subphases.
- Applaunch Setup Run:
During this phase, welcome screens and dialogues during the first launch of the instrumented application are cleared.
- Applaunch Metric Run:
During this phase, the application is launched multiple times determined by the iteration number specified by the parameter
applaunch_iterations
. Each of these iterations are instrumented to capture the launch time taken and the values are recorded as UXPERF marker values in logfile.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- workload_name:
type:
'str'
Name of the uxperf workload to launch
default:
'gmail'
- workload_params:
type:
'OrderedDict'
parameters of the uxperf workload whose application launch time is measured
default:
{ }
- applaunch_type:
type:
'str'
Choose launch_from_long-idle for measuring launch time from long-idle. These two types are described in the workload description.
allowed values:
'launch_from_background'
,'launch_from_long-idle'
default:
'launch_from_background'
- applaunch_iterations:
type:
'integer'
Number of iterations of the application launch
default:
1
benchmarkpi
Measures the time the target device takes to run and complete the Pi calculation algorithm.
http://androidbenchmark.com/howitworks.php
from the website:
The whole idea behind this application is to use the same Pi calculation algorithm on every Android Device and check how fast that process is. Better calculation times, conclude to faster Android devices. This way you can also check how lightweight your custom made Android build is. Or not.
As Pi is an irrational number, Benchmark Pi does not calculate the actual Pi number, but an approximation near the first digits of Pi over the same calculation circles the algorithms needs.
So, the number you are getting in milliseconds is the time your mobile device takes to run and complete the Pi calculation algorithm resulting in a approximation of the first Pi digits.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
chrome
A workload to perform standard Web browsing tasks with Google Chrome. The workload carries out a number of typical Web-based tasks, navigating through a handful of Wikipedia pages in multiple browser tabs.
To run the workload in offline mode, a pages.tar
archive and an
OfflinePages.db
file are required. For users wishing to generate these
files themselves, Chrome should first be operated from an Internet-connected
environment and the following Wikipedia pages should be downloaded for
offline use within Chrome:
Following this, the files of interest for viewing these pages offline can be
found in the /data/data/com.android.chrome/app_chrome/Default/Offline
Pages
directory. The OfflinePages.db
file can be copied from the
‘metadata’ subdirectory, while the *.mhtml
files that should make up the
pages.tar
file can be found in the ‘archives’ subdirectory. These page
files can then be archived to produce a tarball using a command such as
tar -cvf pages.tar -C /path/to/archives .
. Both this and
OfflinePages.db
should then be placed in the
~/.workload_automation/dependencies/chrome/
directory on your local
machine, creating this if it does not already exist.
Known working APK version: 65.0.3325.109
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- offline_mode:
type:
'boolean'
If set to
True
, the workload will execute in offline mode. This mode requires root and makes use of a tarball of *.mhtml files ‘pages.tar’ and an metadata database ‘OfflinePages.db’. The tarball is extracted directly to the application’s offline pages ‘archives’ directory, while the database is copied to the offline pages ‘metadata’ directory.default:
False
deepbench
Benchmarks operations that are important to deep learning. Including GEMM and convolution.
The benchmark and its documentation are available here:
Note
parameters of matrices used in each sub-test are added as classifiers to the metrics. See the benchmark documentation for the explanation of the various parameters
Note
at the moment only the “Arm Benchmarks” subset of DeepBench is supported.
aliases
- deep-gemm
test=``’gemm’``
- deep-conv
test=``’conv’``
- deep-sparse
test=``’sparse’``
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- test:
type:
'str'
Specifies which of the available benchmarks will be run.
- gemm
Performs GEneral Matrix Multiplication of dense matrices of varying sizes.
- conv
Performs convolutions on inputs in NCHW format.
- sparse
Performs GEneral Matrix Multiplication of sparse matrices of varying sizes, and compares them to corresponding dense operations.
allowed values:
'gemm'
,'conv'
,'sparse'
default:
'gemm'
dhrystone
Runs the Dhrystone benchmark.
Original source from:
http://classes.soe.ucsc.edu/cmpe202/benchmarks/standard/dhrystone.c
This version has been modified to configure duration and the number of threads used.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- duration:
type:
'integer'
The duration, in seconds, for which dhrystone will be executed. Either this or
mloops
should be specified but not both.default:
0
- mloops:
type:
'integer'
Millions of loops to run. Either this or
duration
should be specified, but not both. If neither is specified, this will default ‘ to100
default:
0
- threads:
type:
'integer'
The number of separate dhrystone “threads” that will be forked.
default:
4
- delay:
type:
'integer'
The delay, in seconds, between kicking off of dhrystone threads (if
threads
> 1).default:
0
- cpus:
type:
'cpu_mask'
The processes spawned by dhrystone will be pinned to cores as specified by this parameter. The mask can be specified directly as a mask, as a list of cpus or a sysfs- style string
aliases:
'taskset_mask'
default:
0
exoplayer
Android ExoPlayer
ExoPlayer is the basic video player library that is used by the YouTube android app. The aim of this workload is to test a proxy for YouTube performance on targets where running the real YouTube app is not possible due its dependencies.
ExoPlayer sources: https://github.com/google/ExoPlayer
The ‘demo’ application is used by this workload. It can easily be built by loading the ExoPlayer sources into Android Studio.
Version r2.4.0 built from commit d979469 is known to work
Produces a metric ‘exoplayer_dropped_frames’ - this is the count of frames that Exoplayer itself reports as dropped. This is not the same thing as the dropped frames reported by gfxinfo.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
allowed values:
'2.4'
,'2.5'
,'2.6'
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- duration:
type:
'integer'
Playback duration of the video file. This becomes the duration of the workload. If provided must be shorter than the length of the media.
default:
20
- format:
type:
'str'
Specifies which format video file to play. Default is mov_720p
allowed values:
'mp4_1080p'
,'mov_720p'
,'mov_480p'
,'ogg_128kbps'
- filename:
type:
'str'
The name of the video file to play. This can be either a path to the file anywhere on your file system, or it could be just a name, in which case, the workload will look for it in
/home/docs/.workload_automation/dependencies/exoplayer
Note: either format or filename should be specified, but not both!- force_dependency_push:
type:
'boolean'
If true, video will always be pushed to device, regardless of whether the file is already on the device. Default is
False
.default:
False
- landscape:
type:
'boolean'
Configure the screen in landscape mode, otherwise ensure portrait orientation by default. Default is
False
.default:
False
geekbench
Geekbench provides a comprehensive set of benchmarks engineered to quickly and accurately measure processor and memory performance.
http://www.primatelabs.com/geekbench/ From the website: Designed to make benchmarks easy to run and easy to understand, Geekbench takes the guesswork out of producing robust and reliable benchmark results. Geekbench scores are calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are better, with double the score indicating double the performance.
- The benchmarks fall into one of four categories:
integer performance.
floating point performance.
memory performance.
stream performance.
Geekbench benchmarks: http://www.primatelabs.com/geekbench/doc/benchmarks.html Geekbench scoring methedology: http://support.primatelabs.com/kb/geekbench/interpreting-geekbench-scores
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
Specifies which version of the workload should be run.
allowed values:
'6'
,'5'
,'4.4.2'
,'4.4.0'
,'4.3.4'
,'4.3.2'
,'4.3.1'
,'4.2.0'
,'4.0.1'
,'3.4.1'
,'3.0.0'
,'2'
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- loops:
type:
'integer'
Specfies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.
aliases:
'times'
default:
1
- timeout:
type:
'integer'
Timeout for a single iteration of the benchmark. This value is multiplied by
times
to calculate the overall run timeout.default:
3600
- disable_update_result:
type:
'boolean'
If
True
the results file will not be pulled from the targets/data/data/com.primatelabs.geekbench
folder. This allows the workload to be run on unrooted targets and the results extracted manually later.default:
False
geekbench-corporate
Geekbench provides a comprehensive set of benchmarks engineered to quickly and accurately measure processor and memory performance.
http://www.primatelabs.com/geekbench/ From the website: Designed to make benchmarks easy to run and easy to understand, Geekbench takes the guesswork out of producing robust and reliable benchmark results. Geekbench scores are calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are better, with double the score indicating double the performance.
- The benchmarks fall into one of four categories:
integer performance.
floating point performance.
memory performance.
stream performance.
Geekbench benchmarks: http://www.primatelabs.com/geekbench/doc/benchmarks.html Geekbench scoring methedology: http://support.primatelabs.com/kb/geekbench/interpreting-geekbench-scores
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
Specifies which version of the workload should be run.
allowed values:
'5.0.3'
,'5.0.1'
,'4.1.0'
,'4.3.4'
,'5.0.0'
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- loops:
type:
'integer'
Specfies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.
aliases:
'times'
default:
1
- timeout:
type:
'integer'
Timeout for a single iteration of the benchmark. This value is multiplied by
times
to calculate the overall run timeout.default:
3600
- disable_update_result:
type:
'boolean'
If
True
the results file will not be pulled from the targets/data/data/com.primatelabs.geekbench
folder. This allows the workload to be run on unrooted targets and the results extracted manually later.default:
False
gfxbench
Execute a subset of graphical performance benchmarks
Test description: 1. Open the gfxbench application 2. Execute Car Chase, Manhattan and Tessellation benchmarks
- Note: Some of the default tests are unavailable on devices running
with a smaller resolution than 1080p.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- timeout:
type:
'integer'
Timeout for an iteration of the benchmark.
default:
3600
- tests:
type:
'list_or_string'
List of tests to be executed.
default:
['Car Chase', '1080p Car Chase Offscreen', 'Manhattan 3.1', '1080p Manhattan 3.1 Offscreen', '1440p Manhattan 3.1.1 Offscreen', 'Tessellation', '1080p Tessellation Offscreen']
gfxbench-corporate
Execute a subset of graphical performance benchmarks
Test description: 1. Open the gfxbench application 2. Execute Car Chase, Manhattan and Tessellation benchmarks
- Note: Some of the default tests are unavailable on devices running
with a smaller resolution than 1080p.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- timeout:
type:
'integer'
Timeout for an iteration of the benchmark.
default:
3600
- tests:
type:
'list_or_string'
List of tests to be executed.
default:
['Car Chase', '1080p Car Chase Offscreen', 'Manhattan 3.1', '1080p Manhattan 3.1 Offscreen', '1440p Manhattan 3.1.1 Offscreen', 'Tessellation', '1080p Tessellation Offscreen']
glbenchmark
Measures the graphics performance of Android devices by testing the underlying OpenGL (ES) implementation.
http://gfxbench.com/about-gfxbench.jsp
From the website:
The benchmark includes console-quality high-level 3D animations (T-Rex HD and Egypt HD) and low-level graphics measurements.
With high vertex count and complex effects such as motion blur, parallax mapping and particle systems, the engine of GFXBench stresses GPUs in order provide users a realistic feedback on their device.
aliases
glbench
- egypt
use_case=``’egypt’``
- t-rex
use_case=``’t-rex’``
- egypt_onscreen
use_case=``’egypt’
, type=
’onscreen’``- t-rex_onscreen
use_case=``’t-rex’
, type=
’onscreen’``- egypt_offscreen
use_case=``’egypt’
, type=
’offscreen’``- t-rex_offscreen
use_case=``’t-rex’
, type=
’offscreen’``
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
Specifies which version of the benchmark to run (different versions support different use cases).
allowed values:
'2.7'
,'2.5'
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- use_case:
type:
'str'
Specifies which usecase to run, as listed in the benchmark menu; e.g.
'GLBenchmark 2.5 Egypt HD'
. For convenience, two aliases are provided for the most common use cases:'egypt'
and't-rex'
. These could be use instead of the full use case title. For version'2.7'
it defaults to't-rex'
, for version'2.5'
it defaults to'egypt-classic'
.- type:
type:
'str'
Specifies which type of the use case to run, as listed in the benchmarks menu (small text underneath the use case name); e.g.
'C24Z16 Onscreen Auto'
. For convenience, two aliases are provided for the most common types:'onscreen'
and'offscreen'
. These may be used instead of full type names.default:
'onscreen'
- timeout:
type:
'integer'
Specifies how long, in seconds, UI automation will wait for results screen to appear before assuming something went wrong.
default:
200
gmail
A workload to perform standard productivity tasks within Gmail. The workload carries out various tasks, such as creating new emails, attaching images and sending them.
Test description: 1. Open Gmail application 2. Click to create New mail 3. Attach an image from the local images folder to the email 4. Enter recipient details in the To field 5. Enter text in the Subject field 6. Enter text in the Compose field 7. Click the Send mail button
To run the workload in offline mode, a ‘mailstore.tar’ file is required. In order to
generate such a file, Gmail should first be operated from an Internet-connected environment.
After this, the relevant database files can be found in the
‘/data/data/com.google.android.gm/databases’ directory. These files can then be archived to
produce a tarball using a command such as tar -cvf mailstore.tar -C /path/to/databases .
.
The result should then be placed in the ‘~/.workload_automation/dependencies/gmail/’ directory
on your local machine, creating this if it does not already exist.
Known working APK version: 2019.05.26.252424914.release
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- recipient:
type:
'str'
The email address of the recipient. Setting a void address will stop any mesage failures clogging up your device inbox
default:
'wa-devnull@mailinator.com'
- test_image:
type:
'str'
An image to be copied onto the device that will be attached to the email
default:
'uxperf_1600x1200.jpg'
- offline_mode:
type:
'boolean'
If set to
True
, the workload will execute in offline mode. This mode requires root and makes use of a tarball of email database files ‘mailstore.tar’ for the email account to be used. This file is extracted directly to the application’s ‘databases’ directory at ‘/data/data/com.google.android.gm/databases’.default:
False
googlemaps
A workload to perform standard navigation tasks with Google Maps. This workload searches for known locations, pans and zooms around the map, and follows driving directions along a route.
To run the workload in offline mode, databases.tar
and files.tar
archives are required.
In order to generate these files, Google Maps should first be operated from an
Internet-connected environment, and a region around Cambridge, England should be downloaded
for offline use. This region must include the landmarks used in the UIAutomator program,
which include Cambridge train station and Corpus Christi college.
Following this, the files of interest can be found in the databases
and files
subdirectories of the /data/data/com.google.android.apps.maps/
directory. The contents
of these subdirectories can be archived into tarballs using commands such as
tar -cvf databases.tar -C /path/to/databases .
. These databases.tar
and files.tar
archives
should then be placed in the ~/.workload_automation/dependencies/googlemaps
directory on your
local machine, creating this if it does not already exist.
Known working APK version: 10.19.1
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- offline_mode:
type:
'boolean'
If set to
True
, the workload will execute in offline mode. This mode requires root and makes use of a tarball of database filesdatabases.tar
and a tarball of auxiliary filesfiles.tar
. These tarballs are extracted directly to the application’sdatabases
andfiles
directories respectively in/data/data/com.google.android.apps.maps/
.default:
False
googlephotos
A workload to perform standard productivity tasks with Google Photos. The workload carries out various tasks, such as browsing images, performing zooms, and post-processing the image.
Test description:
Four images are copied to the target
The application is started in offline access mode
Gestures are performed to pinch zoom in and out of the selected image
The colour of a selected image is edited by selecting the colour menu, incrementing the colour, resetting the colour and decrementing the colour using the seek bar.
A crop test is performed on a selected image. UiAutomator does not allow the selection of the crop markers so the image is tilted positively, reset and then tilted negatively to get a similar cropping effect.
A rotate test is performed on a selected image, rotating anticlockwise 90 degrees, 180 degrees and 270 degrees.
Known working APK version: 4.53.0.316914374
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- test_images:
type:
'list_of_strs'
A list of four JPEG and/or PNG files to be pushed to the target. Absolute file paths may be used but tilde expansion must be escaped.
constraint:
len(unique(value)) == 4
default:
['uxperf_1200x1600.png', 'uxperf_1600x1200.jpg', 'uxperf_2448x3264.png', 'uxperf_3264x2448.jpg']
googleplaybooks
A workload to perform standard productivity tasks with googleplaybooks. This workload performs various tasks, such as searching for a book title online, browsing through a book, adding and removing notes, word searching, and querying information about the book.
Test description: 1. Open Google Play Books application 2. Dismisses sync operation (if applicable) 3. Searches for a book title 4. Adds books to library if not already present 5. Opens ‘My Library’ contents 6. Opens selected book 7. Gestures are performed to swipe between pages and pinch zoom in and out of a page 8. Selects a specified chapter based on page number from the navigation view 9. Selects a word in the centre of screen and adds a test note to the page 10. Removes the test note from the page (clean up) 11. Searches for the number of occurrences of a common word throughout the book 12. Switches page styles from ‘Day’ to ‘Night’ to ‘Sepia’ and back to ‘Day’ 13. Uses the ‘About this book’ facility on the currently selected book
- NOTE: This workload requires a network connection (ideally, wifi) to run,
a Google account to be setup on the device, and payment details for the account. Free books require payment details to have been setup otherwise it fails. Tip: Install the ‘Google Opinion Rewards’ app to bypass the need to enter valid card/bank detail.
Known working APK version: 3.15.5
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- search_book_title:
type:
'str'
The book title to search for within Google Play Books archive. The book must either be already in the account’s library, or free to purchase.
default:
'Nikola Tesla: Imagination and the Man That Invented the 20th Century'
- library_book_title:
type:
'str'
The book title to search for within My Library. The Library name can differ (usually shorter) to the Store name. If left blank, the
search_book_title
will be used.default:
'Nikola Tesla'
- select_chapter_page_number:
type:
'integer'
The Page Number to search for within a selected book’s Chapter list. Note: Accepts integers only.
default:
4
- search_word:
type:
'str'
The word to search for within a selected book. Note: Accepts single words only.
default:
'the'
- account:
type:
'str'
If you are running this workload on a device which has more than one Google account setup, then this parameter is used to select which account to select when prompted. The account requires the book to have already been purchased or payment details already associated with the account. If omitted, the first account in the list will be selected if prompted.
googleslides
A workload to perform standard productivity tasks with Google Slides. The workload carries out various tasks, such as creating a new presentation, adding text, images, and shapes, as well as basic editing and playing a slideshow. This workload should be able to run without a network connection.
- There are two main scenarios:
create test: a presentation is created in-app and some editing done on it,
load test: a pre-existing PowerPoint file is copied onto the device for testing.
— create —
Create a new file in the application and perform basic editing on it. This test also
requires an image file specified by the param test_image
to be copied onto the device.
Test description:
Start the app and skip the welcome screen. Dismiss the work offline banner if present.
Go to the app settings page and enables PowerPoint compatibility mode. This allows PowerPoint files to be created inside Google Slides.
Create a new PowerPoint presentation in the app (PPT compatibility mode) with a title slide and save it to device storage.
Insert another slide and to it insert the pushed image by picking it from the gallery.
Insert a final slide and add a shape to it. Resize and drag the shape to modify it.
Finally, navigate back to the documents list.
— load —
Copy a PowerPoint presentation onto the device to test slide navigation. The PowerPoint
file to be copied is given by test_file
.
Test description:
From the documents list (following the create test), open the specified PowerPoint by navigating into device storage and wait for it to be loaded.
A navigation test is performed while the file is in editing mode (i.e. not slideshow). swiping forward to the next slide until
slide_count
swipes are performed.While still in editing mode, the same action is done in the reverse direction back to the first slide.
Enter presentation mode by selecting to play the slideshow.
Swipe forward to play the slideshow, for a maximum number of
slide_count
swipes.Finally, repeat the previous step in the reverse direction while still in presentation mode, navigating back to the first slide.
NOTE: There are known issues with the reliability of this workload on some targets.
It MAY NOT ALWAYS WORK on your device. If you do run into problems, it might help to
set do_text_entry
parameter to False
.
Known working APK version: 1.20.442.04.40
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- test_image:
type:
'str'
An image to be copied onto the device that will be embedded in the PowerPoint file as part of the test.
default:
'uxperf_1600x1200.jpg'
- test_file:
type:
'str'
If specified, the workload will copy the PowerPoint file to be used for testing onto the device. Otherwise, a file will be created inside the app.
default:
'uxperf_test_doc.pptx'
- slide_count:
type:
'integer'
Number of slides in aforementioned local file. Determines number of swipe actions when playing slide show.
default:
5
- do_text_entry:
type:
'boolean'
If set to
True
, will attempt to enter text in the first slide as part of the test. Currently seems to be problematic on some devices, most notably Samsung devices.default:
True
hackbench
Hackbench runs a series of tests for the Linux scheduler.
For details, go to: https://github.com/linux-test-project/ltp/
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- timeout:
type:
'integer'
Expected test duration in seconds.
aliases:
'duration'
default:
30
- datasize:
type:
'integer'
Message size in bytes.
default:
100
- groups:
type:
'integer'
Number of groups.
default:
10
- loops:
type:
'integer'
Number of loops.
default:
100
- fds:
type:
'integer'
Number of file descriptors.
default:
40
- extra_params:
type:
'str'
Extra parameters to pass in. See the hackbench man page or type hackbench –help for list of options.
default:
''
homescreen
A workload that goes to the home screen and idles for the the specified duration.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- duration:
type:
'integer'
Specifies the duration, in seconds, of this workload.
default:
20
hwuitest
Tests UI rendering latency on Android devices.
The binary for this workload is built as part of AOSP’s frameworks/base/libs/hwui component.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- test:
type:
'caseless_string'
The test to run:
'shadowgrid'
: creates a grid of rounded rects that cast shadows, high CPU & GPU load'rectgrid'
: creates a grid of 1x1 rects'oval'
: draws 1 oval
allowed values:
'shadowgrid'
,'rectgrid'
,'oval'
default:
'shadowgrid'
- loops:
type:
'integer'
The number of test iterations.
default:
3
- frames:
type:
'integer'
The number of frames to run the test over.
default:
150
idle
Do nothing for the specified duration.
On android devices, this may optionally stop the Android run time, if
stop_android
is set to True
.
Note
This workload requires the device to be rooted.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- duration:
type:
'integer'
Specifies the duration, in seconds, of this workload.
default:
20
- screen_off:
type:
'boolean'
Ensure that the screen is off before idling.
Note
Make sure screen lock is disabled on the target!
default:
False
- stop_android:
type:
'boolean'
Specifies whether the Android run time should be stopped. (Can be set only for Android devices).
default:
False
jankbench
Internal Google benchmark for evaluating jank on Android.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- test_ids:
type:
'list_or_string'
ID of the jankbench test to be run.
allowed values:
'list_view'
,'image_list_view'
,'shadow_grid'
,'low_hitrate_text'
,'high_hitrate_text'
,'edit_text'
,'overdraw_test'
- loops:
type:
'integer'
Specifies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.
aliases:
'reps'
constraint:
value > 0
default:
1
- pull_results_db:
type:
'boolean'
Secifies whether an sqlite database with detailed results should be pulled from benchmark app’s data. This requires the device to be rooted.
This defaults to
True
for rooted devices andFalse
otherwise.- timeout:
type:
'integer'
Time out for workload execution. The workload will be killed if it hasn’t completed within this period.
aliases:
'run_timeout'
default:
600
lmbench
Run a subtest from lmbench, a suite of portable ANSI/C microbenchmarks for UNIX/POSIX. In general, lmbench measures two key features: latency and bandwidth. This workload supports a subset of lmbench tests. lat_mem_rd can be used to measure latencies to memory (including caches). bw_mem can be used to measure bandwidth to/from memory over a range of operations. Further details, and source code are available from:
See lmbench/bin/README for license details.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- test:
type:
'str'
Specifies an lmbench test to run.
allowed values:
'lat_mem_rd'
,'bw_mem'
default:
'lat_mem_rd'
- stride:
type:
'<locals'
Stride for lat_mem_rd test. Workload will iterate over one or more integer values.
default:
[128]
- thrash:
type:
'boolean'
Sets -t flag for lat_mem_rd_test
default:
True
- size:
type:
'list_or_string'
Data set size for lat_mem_rd bw_mem tests.
default:
'4m'
- mem_category:
type:
'list_or_string'
List of memory catetories for bw_mem test.
default:
('rd', 'wr', 'cp', 'frd', 'fwr', 'fcp', 'bzero', 'bcopy')
- parallelism:
type:
'integer'
Parallelism flag for tests that accept it.
- warmup:
type:
'integer'
Warmup flag for tests that accept it.
- repetitions:
type:
'integer'
Repetitions flag for tests that accept it.
- force_abi:
type:
'str'
Override device abi with this value. Can be used to force arm32 on 64-bit devices.
- run_timeout:
type:
'integer'
Timeout for execution of the test.
default:
900
- loops:
type:
'integer'
Specifies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween. This parameter is distinct from “repetitions”, as the latter takes place within the benchmark and produces a single result.
constraint:
value > 0
default:
1
- cpus:
type:
'cpu_mask'
Specifies the CPU mask the benchmark process will be pinned to.
aliases:
'taskset_mask'
default:
0
manual
Yields control to the user, either for a fixed period or based on user input, to perform custom operations on the device, which workload automation does not know of.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- duration:
type:
'integer'
Control of the devices is yielded for the duration (in seconds) specified. If not specified,
user_triggered
is assumed.- user_triggered:
type:
'boolean'
If
True
, WA will wait for user input after starting the workload; otherwise fixed duration is expected. Defaults toTrue
ifduration
is not specified, andFalse
otherwise.- view:
type:
'str'
Specifies the View of the workload. This enables instruments that require a View to be specified, such as the
fps
instrument. This is required for using “SurfaceFlinger” to collect FPS statistics and is primarily used on devices pre API level 23.default:
'SurfaceView'
- package:
type:
'str'
Specifies the package name of the workload. This enables instruments that require a Package to be specified, such as the
fps
instrument. This allows for “gfxinfo” to be used and is the preferred method of collection for FPS statistics on devices API level 23+.
meabo
A multi-phased multi-purpose micro-benchmark. The micro-benchmark is composed of 10 phases that perform various generic calculations (from memory to compute intensive).
It is a highly configurable tool which can be used for energy efficiency studies, ARM big.LITTLE Linux scheduler analysis and DVFS studies. It can be used for other benchmarking as well.
All floating-point calculations are double-precision.
For more details and benchmark source, see:
Note
current implementation of automation relies on the executable to be either statically linked or for all necessary depencies to be installed on the target.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- array_size:
type:
'integer'
Size of arrays used in Phases 1, 2, 3, 4 and 5.
constraint:
value > 0
default:
1048576
- num_rows:
type:
'integer'
Number of rows for the sparse matrix used in Phase 6.
aliases:
'nrow'
constraint:
value > 0
default:
16384
- num_cols:
type:
'integer'
Number of columns for the sparse matrix used in Phase 6.
aliases:
'ncol'
constraint:
value > 0
default:
16384
- loops:
type:
'integer'
Number of iterations that core loop is executed.
aliases:
'num_iterations'
constraint:
value > 0
default:
1000
- block_size:
type:
'integer'
Block size used in Phase 1.
constraint:
value > 0
default:
8
- num_cpus:
type:
'integer'
Number of total CPUs that the application can bind threads to.
constraint:
value > 0
default:
6
- per_phase_cpu_ids:
type:
'list_of_ints'
Sets which cores each phase is run on.
constraint:
all(v >= -1 for v in value)
default:
[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
- num_hwcntrs:
type:
'integer'
Only available when using PAPI. Controls how many hardware counters PAPI will get access to.
constraint:
value >= 0
default:
7
- run_phases:
type:
'list_of_ints'
Controls which phases to run.
constraint:
all(0 < v <= 10 for v in value)
default:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
- threads:
type:
'integer'
Controls how many threads the application will be using.
aliases:
'num_threads'
constraint:
value >= 0
default:
0
- bind_to_cpu_set:
type:
'integer'
Controls whether threads will be bound to a core set, or each individual thread will be bound to a specific core within the core set.
constraint:
0 <= value <= 1
default:
1
- llist_size:
type:
'integer'
Size of the linked list available for each thread.
constraint:
value > 0
default:
16777216
- num_particles:
type:
'integer'
Number of particles used in Phase 8.
constraint:
value > 0
default:
1048576
- num_palindromes:
type:
'integer'
Number of palindromes used in Phase 9.
constraint:
value > 0
default:
1024
- num_randomloc:
type:
'integer'
Number of random memory locations accessed in Phase 10.
constraint:
value > 0
default:
2097152
- timeout:
type:
'integer'
Timeout for execution of the test.
aliases:
'run_timeout'
constraint:
value > 0
default:
2700
memcpy
Runs memcpy in a loop.
This will run memcpy in a loop for a specified number of times on a buffer of a specified size. Additionally, the affinity of the test can be set to one or more specific cores.
This workload is single-threaded. It generates no scores or metrics by itself.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- buffer_size:
type:
'integer'
Specifies the size, in bytes, of the buffer to be copied.
default:
5242880
- loops:
type:
'integer'
Specfies the number of iterations that will be performed.
aliases:
'iterations'
default:
1000
- cpus:
type:
'cpu_mask'
The cpus for which the affinity of the test process should be set, specified as a mask, as a list of cpus or a sysfs-style string. If not specified, all available cores will be used.
default:
0
mongoperf
A utility to check disk I/O performance independently of MongoDB.
It times tests of random disk I/O and presents the results. You can use mongoperf for any case apart from MongoDB. The mmf true mode is completely generic.
Note
mongoperf
seems to ramp up threads in powers of two over a
period of tens of seconds (there doesn’t appear to be a way to
change that). Bear this in mind when setting the duration
.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- duration:
type:
'integer'
Duration of of the workload.
default:
300
- threads:
type:
'integer'
Defines the number of threads mongoperf will use in the test. To saturate the system storage system you will need multiple threads.
default:
16
- file_size_mb:
type:
'integer'
Test file size in MB.
default:
1
- sleep_micros:
type:
'integer'
mongoperf will pause for this number of microseconds divided by the the number of threads between each operation.
default:
0
- mmf:
type:
'boolean'
When
True
, use memory mapped files for the tests. Generally:when mmf is
False
, mongoperf tests direct, physical, I/O, without caching. Use a large file size to test heavy random I/O load and to avoid I/O coalescing.when mmf is
True
, mongoperf runs tests of the caching system, and can use normal file system cache. Use mmf in this mode to test file system cache behavior with memory mapped files.
default:
True
- read:
type:
'boolean'
When
True
, perform reads as part of the test. Eitherread
orwrite
must beTrue
.aliases:
'r'
default:
True
- write:
type:
'boolean'
When
True
, perform writes as part of the test. Eitherread
orwrite
must beTrue
.aliases:
'w'
default:
True
- rec_size_kb:
type:
'integer'
The size of each write operation
default:
4
- sync_delay:
type:
'integer'
Seconds between disk flushes. Only use this if
mmf
is set toTrue
.default:
0
motionmark
A workload to execute the motionmark web based benchmark
MotionMark is a graphics benchmark that measures a browser capability to animate complex scenes at a target frame rate
Test description: 1. Open browser application 2. Navigate to the motionmark website - http://browserbench.org/MotionMark/ 3. Execute the benchmark
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
openssl
Benchmark Openssl algorithms using Openssl’s speed command.
The command tests how long it takes to perfrom typical SSL operations using a range of supported algorithms and ciphers.
By defalt, this workload will use openssl installed on the target, however it is possible to provide an alternative binary as a workload resource.
aliases
- ossl-aes-128-cbc
algorithm=``’aes-128-cbc’``
- ossl-aes-192-cbc
algorithm=``’aes-192-cbc’``
- ossl-aes-256-cbc
algorithm=``’aes-256-cbc’``
- ossl-aes-128-gcm
algorithm=``’aes-128-gcm’``
- ossl-aes-192-gcm
algorithm=``’aes-192-gcm’``
- ossl-aes-256-gcm
algorithm=``’aes-256-gcm’``
- ossl-sha1
algorithm=``’sha1’``
- ossl-sha256
algorithm=``’sha256’``
- ossl-sha384
algorithm=``’sha384’``
- ossl-sha512
algorithm=``’sha512’``
- ossl-rsa
algorithm=``’rsa’``
- ossl-dsa
algorithm=``’dsa’``
- ossl-ecdh
algorithm=``’ecdh’``
- ossl-ecdsa
algorithm=``’ecdsa’``
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- algorithm:
type:
'str'
Algorithm to benchmark.
allowed values:
'aes-128-cbc'
,'aes-192-cbc'
,'aes-256-cbc'
,'aes-128-gcm'
,'aes-192-gcm'
,'aes-256-gcm'
,'sha1'
,'sha256'
,'sha384'
,'sha512'
,'rsa'
,'dsa'
,'ecdh'
,'ecdsa'
default:
'aes-256-cbc'
- threads:
type:
'integer'
The number of threads to use
default:
1
- use_system_binary:
type:
'boolean'
If
True
, the system Openssl binary will be used. Otherwise, use the binary provided in the workload resources.default:
True
pcmark
A workload to execute the Work x.0 benchmarks within PCMark - https://www.futuremark.com/benchmarks/pcmark-android
Test description: 1. Open PCMark application 2. Swipe right to the Benchmarks screen 3. Select the Work x.0 benchmark 4. If needed, install the Work x.0 benchmark (requires an internet connection) 5. Execute the Work x.0 benchmark
Known working APK versions: 3.0.4061, 2.0.3716
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
Specifies which version of the workload should be run.
allowed values:
'3'
,'2'
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
recentfling
Tests UI jank on android devices.
For this workload to work, recentfling.sh
and defs.sh
must be placed
in ~/.workload_automation/dependencies/recentfling/
. These can be found
in the AOSP Git repository.
To change the apps that are opened at the start of the workload you will need
to modify the defs.sh
file. You will need to add your app to dfltAppList
and then add a variable called {app_name}Activity
with the name of the
activity to launch (where {add_name}
is the name you put into dfltAppList
).
You can get a list of activities available on your device by running
adb shell pm list packages -f
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- loops:
type:
'integer'
The number of test iterations.
default:
3
- start_apps:
type:
'boolean'
If set to
False
,no apps will be started before flinging through the recent apps list (in which the assumption is there are already recently started apps in the list.default:
True
- device_name:
type:
'str'
If set, recentfling will use the fling parameters for this device instead of automatically guessing the device. This can also be used if the device is not supported by recentfling, but its screensize is similar to that of one that is supported.
For possible values, check your recentfling.sh. At the time of writing, valid values are: ‘shamu’, ‘hammerhead’, ‘angler’, ‘ariel’, ‘mtp8996’, ‘bullhead’ or ‘volantis’.
rt-app
A test application that simulates configurable real-time periodic load.
rt-app is a test application that starts multiple periodic threads in order to simulate a real-time periodic load. It supports SCHED_OTHER, SCHED_FIFO, SCHED_RR as well as the AQuoSA framework and SCHED_DEADLINE.
The load is described using JSON-like config files. Below are a couple of simple examples.
Simple use case which creates a thread that run 1ms then sleep 9ms until the use case is stopped with Ctrl+C:
{
"tasks" : {
"thread0" : {
"loop" : -1,
"run" : 20000,
"sleep" : 80000
}
},
"global" : {
"duration" : 2,
"calibration" : "CPU0",
"default_policy" : "SCHED_OTHER",
"pi_enabled" : false,
"lock_pages" : false,
"logdir" : "./",
"log_basename" : "rt-app1",
"ftrace" : false,
"gnuplot" : true,
}
}
Simple use case with 2 threads that runs for 10 ms and wake up each other until the use case is stopped with Ctrl+C
{
"tasks" : {
"thread0" : {
"loop" : -1,
"run" : 10000,
"resume" : "thread1",
"suspend" : "thread0"
},
"thread1" : {
"loop" : -1,
"run" : 10000,
"resume" : "thread0",
"suspend" : "thread1"
}
}
}
Please refer to the existing configs in $WA_ROOT/wa/workloads/rt_app/use_case
for more examples.
The upstream version of rt-app is hosted here:
https://github.com/scheduler-tools/rt-app
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If set to
True
, rt-app binary will be uninstalled from the device at the end of the run.aliases:
'uninstall_on_exit'
default:
False
- config:
type:
'str'
Use case configuration file to run with rt-app. This may be either the name of one of the “standard” configurations included with the workload. or a path to a custom JSON file provided by the user. Either way, the “.json” extension is implied and will be added automatically if not specified in the argument.
The following is the list of standard configurations currently included with the workload: camera-short.json, camera-long.json, browser-short.json, video-long.json, mp3-long.json, spreading-tasks.json, browser-long.json, mp3-short.json, taskset.json, video-short.json
default:
'taskset'
- duration:
type:
'integer'
Duration of the workload execution in Seconds. If specified, this will override the corresponding parameter in the JSON config.
- cpus:
type:
'cpu_mask'
Constrain execution to specific CPUs.
aliases:
'taskset_mask'
default:
0
- force_install:
type:
'boolean'
If set to
True
, rt-app binary will always be deployed to the target device at the beginning of the run, regardless of whether it was already installed there.default:
False
schbench
Benchmark providing detailed latency distribution statistics for scheduler wakeups.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- runtime:
type:
'integer'
How long to run before exiting (seconds)
aliases:
'timeout'
default:
30
- message_threads:
type:
'integer'
number of message threads
default:
2
- threads:
type:
'integer'
worker threads per message thread
default:
16
- sleeptime:
type:
'integer'
Message thread latency (usec)
default:
30000
- cputime:
type:
'integer'
How long to think during loop
default:
30000
- auto:
type:
'boolean'
grow thread count until latencies hurt
default:
False
- pipe:
type:
'integer'
transfer size bytes to simulate a pipe test
default:
0
- rps:
type:
'integer'
requests per second mode (count)
default:
0
- intervaltime:
type:
'integer'
interval for printing latencies (seconds)
shellscript
Runs an arbitrary shellscript on the target.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- script_file: (mandatory)
type:
'str'
The path (on the host) to the shell script file. This must be an absolute path (though it may contain ~).
- argstring:
type:
'str'
A string that should contain arguments passed to the script.
default:
''
- as_root:
type:
'boolean'
Specify whether the script should be run as root.
default:
False
- timeout:
type:
'integer'
Timeout, in seconds, for the script run time.
default:
60
speedometer
A workload to execute the speedometer 2.0 web based benchmark. Requires device to be rooted. This workload will only with Android 9+ devices if connected via TCP, or Android 5+ if connected via USB.
Test description:
Host a local copy of the Speedometer website, and make it visible to the device via ADB.
Open chrome via an intent to access the local copy.
Execute the benchmark - the copy has been modified to trigger the start of the benchmark.
The benchmark will write to the browser’s sandboxed local storage to signal the benchmark has completed. This local storage is monitored by this workload.
Known working chrome version 83.0.4103.106
To modify the archived speedometer workload:
Run ‘git clone https://github.com/WebKit/webkit’
Copy PerformanceTests/Speedometer to a directory called document_root, renaming Speedometer to Speedometer2.0
Modify document_root/Speedometer2.0/index.html:
3a. Remove the ‘defer’ attribute from the <script> tags within the <head> section. 3b. Add ‘<script>startTest();</script>’ to the very end of the <body> section.
Modify document_root/Speedometer2.0/resources/main.js:
4a. Add the listed code after this line:
document.getElementById(‘result-number’).textContent = results.formattedMean;
Code to add:
if (location.search.length > 1) { var parts = location.search.substring(1).split('&'); for (var i = 0; i < parts.length; i++) { var keyValue = parts[i].split('='); var key = keyValue[0]; var value = keyValue[1]; if (key === "reportEndId") { window.localStorage.setItem('reportEndId', value); } } }
Run ‘tar -cpzf speedometer_archive.tgz document_root’
Copy the tarball into the workloads/speedometer directory
If appropriate, update the commit info in the LICENSE file.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- chrome_package:
type:
'str'
The app package for the browser that will be launched.
allowed values:
'org.chromium.chrome'
,'com.android.chrome'
default:
'com.android.chrome'
stress-ng
Run the stress-ng benchmark.
stress-ng will stress test a computer system in various selectable ways. It was designed to exercise various physical subsystems of a computer as well as the various operating system kernel interfaces.
stress-ng can also measure test throughput rates; this can be useful to observe performance changes across different operating system releases or types of hardware. However, it has never been intended to be used as a precise benchmark test suite, so do NOT use it in this manner.
- The official website for stress-ng is at:
- Source code are available from:
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- stressor:
type:
'str'
Stress test case name. The cases listed in allowed values come from the stable release version 0.01.32. The binary included here compiled from dev version 0.06.01. Refer to man page for the definition of each stressor.
allowed values:
'cpu'
,'io'
,'fork'
,'switch'
,'vm'
,'pipe'
,'yield'
,'hdd'
,'cache'
,'sock'
,'fallocate'
,'flock'
,'affinity'
,'timer'
,'dentry'
,'urandom'
,'sem'
,'open'
,'sigq'
,'poll'
default:
'cpu'
- extra_args:
type:
'str'
Extra arguments to pass to the workload.
Please note that these are not checked for validity.
default:
''
- threads:
type:
'integer'
The number of workers to run. Specifying a negative or zero value will select the number of online processors.
default:
0
- duration:
type:
'integer'
Timeout for test execution in seconds
default:
60
sysbench
A modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load.
The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.
Features of SysBench
file I/O performance
scheduler performance
memory allocation and transfer speed
POSIX threads implementation performance
database server performance
See: https://github.com/akopytov/sysbench
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, executables that are installed to the device as part of the workload will be uninstalled again.default:
True
- timeout:
type:
'integer'
timeout for workload execution (adjust from default if running on a slow target and/or specifying a large value for
max_requests
default:
300
- test:
type:
'str'
sysbench test to run
allowed values:
'fileio'
,'cpu'
,'memory'
,'threads'
,'mutex'
default:
'cpu'
- threads:
type:
'integer'
The number of threads sysbench will launch.
aliases:
'num_threads'
default:
8
- max_requests:
type:
'integer'
The limit for the total number of requests.
- max_time:
type:
'integer'
The limit for the total execution time. If neither this nor
max_requests
is specified, this will default to 30 seconds.- file_test_mode:
type:
'str'
File test mode to use. This should only be specified if
test
is"fileio"
; if that is the case andfile_test_mode
is not specified, it will default to"seqwr"
(please see sysbench documentation for explanation of various modes).allowed values:
'seqwr'
,'seqrewr'
,'seqrd'
,'rndrd'
,'rndwr'
,'rndrw'
- cmd_params:
type:
'str'
Additional parameters to be passed to sysbench as a single string.
default:
''
- cpus:
type:
'cpu_mask'
The processes spawned by sysbench will be pinned to cores as specified by this parameter. Can be provided as a mask, a list of cpus or a sysfs-style string.
aliases:
'taskset_mask'
default:
0
templerun2
Temple Run 2 game.
Sequel to Temple Run. 3D on-the-rails racer.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.
thechase
The Chase demo showcasing the capabilities of Unity game engine.
This demo, is a static video-like game demo, that demonstrates advanced features of the unity game engine. It loops continuously until terminated.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- duration:
type:
'integer'
Duration, in seconds, note that the demo loops the same (roughly) 60 second sceene until stopped.
default:
70
uibench
Runs a particular activity of the UIBench workload suite. The suite is provided by Google as a testbench for the Android UI.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- activity:
type:
'str'
The UIBench activity to be run. Each activity corresponds to a test. If this parameter is ignored, the application is launched in its main menu. Please note that the available activities vary between versions of UIBench (which follow AOSP versioning) and the availability of the services under test may depend on the version of the target Android. We recommend using the APK of UIBench corresponding to the Android version, enforced through the
version
parameter to this workload.- duration:
type:
'integer'
As activities do not finish, this workload will terminate UIBench after the given duration.
default:
10
uibenchjanktests
Runs a particular test of the UIBench JankTests test suite. The suite is provided by Google as an automated version of the UIBench testbench for the Android UI.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- test:
type:
'str'
Test to be run. Defaults to full run.
- wait:
type:
'boolean'
Forces am instrument to wait until the instrumentation terminates before terminating itself. The net effect is to keep the shell open until the tests have finished. This flag is not required, but if you do not use it, you will not see the results of your tests.
default:
True
- raw:
type:
'boolean'
Outputs results in raw format. Use this flag when you want to collect performance measurements, so that they are not formatted as test results. This flag is designed for use with the flag -e perf true.
default:
False
- instrument_args:
type:
'OrderedDict'
Extra arguments for am instrument.
default:
{ }
- no_hidden_api_checks:
type:
'boolean'
Disables restrictions on the use of hidden APIs.
default:
False
vellamo
Android benchmark designed by Qualcomm.
Vellamo began as a mobile web benchmarking tool that today has expanded to include three primary chapters. The Browser Chapter evaluates mobile web browser performance, the Multicore chapter measures the synergy of multiple CPU cores, and the Metal Chapter measures the CPU subsystem performance of mobile processors. Through click-and-go test suites, organized by chapter, Vellamo is designed to evaluate: UX, 3D graphics, and memory read/write and peak bandwidth performance, and much more!
Note: Vellamo v3.0 fails to run on Juno
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
Specify the version of Vellamo to be run. If not specified, the latest available version will be used.
allowed values:
'2.0.3'
,'3.0'
,'3.2.4'
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- benchmarks:
type:
'list_of_strs'
Specify which benchmark sections of Vellamo to be run. Only valid on version 3.0 and newer. NOTE: Browser benchmark can be problematic and seem to hang,just wait and it will progress after ~5 minutes
allowed values:
'Browser'
,'Metal'
,'Multi'
default:
['Browser', 'Metal', 'Multi']
- browser:
type:
'integer'
Specify which of the installed browsers will be used for the tests. The number refers to the order in which browsers are listed by Vellamo. E.g.
1
will select the first browser listed,2
– the second, etc. Only valid for version3.0
.default:
1
youtube
A workload to perform standard productivity tasks within YouTube.
The workload plays a video from the app, determined by the video_source
parameter.
While the video is playing, a some common actions are done such as video seeking, pausing
playback and navigating the comments section.
Test description:
The video_source
parameter determines where the video to be played will be found
in the app. Possible values are search
, home
, my_videos
, and trending
.
- -A. search - Goes to the search view, does a search for the given term, and plays the
first video in the results. The parameter
search_term
must also be provided in the agenda for this to work. This is the default mode.- -B. home - Scrolls down once on the app’s home page to avoid ads (if present, would be
first video), then select and plays the video that appears at the top of the list.
- -C. my_videos - Goes to the ‘My Videos’ section of the user’s account page and plays a
video from there. The user must have at least one uploaded video for this to work.
- -D. trending - Goes to the ‘Trending Videos’ section of the app, and plays the first
video in the trending videos list.
For the selected video source, the following test steps are performed:
Navigate to the general app settings page to disable autoplay. This improves test stability and predictability by preventing screen transition to load a new video while in the middle of the test.
Select the video from the source specified above, and dismiss any potential embedded advert that may pop-up before the actual video.
Let the video play for a few seconds, pause it, then resume.
Expand the info card that shows video metadata, then collapse it again.
Scroll down to the end of related videos and comments under the info card, and then back up to the start. A maximum of 5 swipe actions is performed in either direction.
Known working APK version: 15.45.32
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- markers_enabled:
type:
'boolean'
If set to
True
, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.default:
False
- video_source:
type:
'str'
Determines where to play the video from. This can either be from the YouTube home, my videos section, trending videos or found in search.
allowed values:
'home'
,'my_videos'
,'search'
,'trending'
default:
'search'
- search_term:
type:
'str'
The search term to use when
video_source
is set tosearch
. Ignored otherwise.default:
'Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film'
youtube_playback
Simple Youtube video playback
This triggers a video streaming playback on Youtube. Unlike the more featureful “youtube” workload, this performs no other action that starting the video via an intent and then waiting for a certain amount of playback time. This is therefore only useful when you are confident that the content on the end of the provided URL is stable - that means the video should have no advertisements attached.
parameters
- cleanup_assets:
type:
'boolean'
If
True
, assets that are deployed or created by the plugin will be removed again from the device.aliases:
'clean_up'
global alias:
'cleanup_assets'
default:
True
- uninstall:
type:
'boolean'
If
True
, will uninstall workload’s APK as part of teardown.’default:
False
- package_name:
type:
'str'
The package name that can be used to specify the workload apk to use.
- install_timeout:
type:
'integer'
Timeout for the installation of the apk.
constraint:
value > 0
default:
300
- version:
type:
'str'
The version of the package to be used.
- max_version:
type:
'str'
The maximum version of the package to be used.
- min_version:
type:
'str'
The minimum version of the package to be used.
- variant:
type:
'str'
The variant of the package to be used.
- strict:
type:
'boolean'
Whether to throw an error if the specified package cannot be found on host.
default:
False
- force_install:
type:
'boolean'
Always re-install the APK, even if matching version is found already installed on the device.
default:
False
- exact_abi:
type:
'boolean'
If
True
, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.default:
False
- prefer_host_package:
type:
'boolean'
If
True
then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. IfFalse
then the version on the target is preferred instead.aliases:
'check_apk'
default:
True
- view:
type:
'str'
Manually override the ‘View’ of the workload for use with instruments such as the
fps
instrument. If not specified, a workload dependant ‘View’ will be automatically generated.- video_url:
type:
'str'
URL of video to play
default:
'https://www.youtube.com/watch?v=YE7VzlLtp-4'
- duration:
type:
'integer'
Number of seconds of video to play
default:
20