Workloads

adobereader 

The Adobe Reader workflow carries out the following typical productivity tasks.

Test description:

Open a local file on the device
Gestures test:
2.1. Swipe down across the central 50% of the screen in 200 x 5ms steps 2.2. Swipe up across the central 50% of the screen in 200 x 5ms steps 2.3. Swipe right from the edge of the screen in 50 x 5ms steps 2.4. Swipe left from the edge of the screen in 50 x 5ms steps 2.5. Pinch out 50% in 100 x 5ms steps 2.6. Pinch In 50% in 100 x 5ms steps
Search test:
Search document_name for each string in the search_string_list
Close the document

Known working APK version: 19.7.1.10709

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

document_name:

type: 'str'

The document name to use for the Gesture and Search test.

default: 'uxperf_test_doc.pdf'

search_string_list:

type: 'list_of_strs'

For each string in the list, a document search is performed using the string as the search term. At least one must be provided.

constraint: len(value) > 0

default: ['The quick brown fox jumps over the lazy dog', 'TEST_SEARCH_STRING']

aitutu 

Executes Aitutu Image Speed/Accuracy and Object Speed/Accuracy tests

The Aitutu workflow carries out the following tasks. 1. Open Aitutu application 2. Download the resources for the test 3. Execute the tests

Known working APK version: 1.0.3

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

androbench 

Executes storage performance benchmarks

The Androbench workflow carries out the following typical productivity tasks. 1. Open Androbench application 2. Execute all memory benchmarks

Known working APK version: 5.0.1

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

angrybirds_rio 

Angry Birds Rio game.

The sequel to the very popular Android 2D game.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

antutu 

Executes Antutu 3D, UX, CPU and Memory tests

Test description: 1. Open Antutu application 2. Execute Antutu benchmark

Known working APK version: 8.0.4

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

Specify the version of Antutu to be run. If not specified, the latest available version will be used.

allowed values: '7.0.4', '7.2.0', '8.0.4', '8.1.9', '9.1.6'

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

apache 

Load-test an apache installation by issueing parallel requests with ab.

Run ab, the Apache benchmark on the host, directed at the target as the server.

Note

It is assumed that Apache is already running on target.

Note

Current implmentation only supports a very basic use of the benchmark.

aliases

ab

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

port:

type: 'integer'

Port on which Apache is running.

default: 80

path:

type: 'str'

Path to request.

default: '/'

parallel_requests:

type: 'integer'

The number of parallel requests at a time.

default: 350

total_requests:

type: 'integer'

The total number of parallel requests.

default: 100000

applaunch 

This workload launches and measures the launch time of applications for supporting workloads.

Currently supported workloads are the ones that implement ApplaunchInterface. For any workload to support this workload, it should implement the ApplaunchInterface. The corresponding java file of the workload associated with the application being measured is executed during the run. The application that needs to be measured is passed as a parameter workload_name. The parameters required for that workload have to be passed as a dictionary which is captured by the parameter workload_params. This information can be obtained by inspecting the workload details of the specific workload.

The workload allows to run multiple iterations of an application launch in two modes:

Launch from background
Launch from long-idle

These modes are captured as a parameter applaunch_type.

launch_from_background: Launches an application after the application is sent to background by pressing Home button.
launch_from_long-idle: Launches an application after killing an application process and clearing all the caches.

Test Description:

During the initialization and setup, the application being launched is launched for the first time. The jar file of the workload of the application is moved to device at the location workdir which further implements the methods needed to measure the application launch time.
Run phase calls the UiAutomator of the applaunch which runs in two subphases.
1. Applaunch Setup Run:
  During this phase, welcome screens and dialogues during the first launch of the instrumented application are cleared.
2. Applaunch Metric Run:
  During this phase, the application is launched multiple times determined by the iteration number specified by the parameter applaunch_iterations. Each of these iterations are instrumented to capture the launch time taken and the values are recorded as UXPERF marker values in logfile.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

workload_name:

type: 'str'

Name of the uxperf workload to launch

default: 'gmail'

workload_params:

type: 'OrderedDict'

parameters of the uxperf workload whose application launch time is measured

default:

{

}

applaunch_type:

type: 'str'

Choose launch_from_long-idle for measuring launch time from long-idle. These two types are described in the workload description.

allowed values: 'launch_from_background', 'launch_from_long-idle'

default: 'launch_from_background'

applaunch_iterations:

type: 'integer'

Number of iterations of the application launch

default: 1

benchmarkpi 

Measures the time the target device takes to run and complete the Pi calculation algorithm.

http://androidbenchmark.com/howitworks.php

from the website:

The whole idea behind this application is to use the same Pi calculation algorithm on every Android Device and check how fast that process is. Better calculation times, conclude to faster Android devices. This way you can also check how lightweight your custom made Android build is. Or not.

As Pi is an irrational number, Benchmark Pi does not calculate the actual Pi number, but an approximation near the first digits of Pi over the same calculation circles the algorithms needs.

So, the number you are getting in milliseconds is the time your mobile device takes to run and complete the Pi calculation algorithm resulting in a approximation of the first Pi digits.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

chrome 

A workload to perform standard Web browsing tasks with Google Chrome. The workload carries out a number of typical Web-based tasks, navigating through a handful of Wikipedia pages in multiple browser tabs.

To run the workload in offline mode, a pages.tar archive and an OfflinePages.db file are required. For users wishing to generate these files themselves, Chrome should first be operated from an Internet-connected environment and the following Wikipedia pages should be downloaded for offline use within Chrome:

Following this, the files of interest for viewing these pages offline can be found in the /data/data/com.android.chrome/app_chrome/Default/Offline Pages directory. The OfflinePages.db file can be copied from the ‘metadata’ subdirectory, while the *.mhtml files that should make up the pages.tar file can be found in the ‘archives’ subdirectory. These page files can then be archived to produce a tarball using a command such as tar -cvf pages.tar -C /path/to/archives .. Both this and OfflinePages.db should then be placed in the ~/.workload_automation/dependencies/chrome/ directory on your local machine, creating this if it does not already exist.

Known working APK version: 65.0.3325.109

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

offline_mode:

type: 'boolean'

If set to True, the workload will execute in offline mode. This mode requires root and makes use of a tarball of *.mhtml files ‘pages.tar’ and an metadata database ‘OfflinePages.db’. The tarball is extracted directly to the application’s offline pages ‘archives’ directory, while the database is copied to the offline pages ‘metadata’ directory.

default: False

deepbench 

Benchmarks operations that are important to deep learning. Including GEMM and convolution.

The benchmark and its documentation are available here:

https://github.com/baidu-research/DeepBench

Note

parameters of matrices used in each sub-test are added as classifiers to the metrics. See the benchmark documentation for the explanation of the various parameters

Note

at the moment only the “Arm Benchmarks” subset of DeepBench is supported.

aliases

deep-gemm: test=``’gemm’``
deep-conv: test=``’conv’``
deep-sparse: test=``’sparse’``

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

test:

type: 'str'

Specifies which of the available benchmarks will be run.

gemm: Performs GEneral Matrix Multiplication of dense matrices of varying sizes.
conv: Performs convolutions on inputs in NCHW format.
sparse: Performs GEneral Matrix Multiplication of sparse matrices of varying sizes, and compares them to corresponding dense operations.

allowed values: 'gemm', 'conv', 'sparse'

default: 'gemm'

dhrystone 

Runs the Dhrystone benchmark.

Original source from:

http://classes.soe.ucsc.edu/cmpe202/benchmarks/standard/dhrystone.c

This version has been modified to configure duration and the number of threads used.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

duration:

type: 'integer'

The duration, in seconds, for which dhrystone will be executed. Either this or mloops should be specified but not both.

default: 0

mloops:

type: 'integer'

Millions of loops to run. Either this or duration should be specified, but not both. If neither is specified, this will default ‘ to 100

default: 0

threads:

type: 'integer'

The number of separate dhrystone “threads” that will be forked.

default: 4

delay:

type: 'integer'

The delay, in seconds, between kicking off of dhrystone threads (if threads > 1).

default: 0

cpus:

type: 'cpu_mask'

The processes spawned by dhrystone will be pinned to cores as specified by this parameter. The mask can be specified directly as a mask, as a list of cpus or a sysfs- style string

aliases: 'taskset_mask'

default: 0

drarm 

Dr. Arm’s Amazing Adventures is a “Souls-Like” Mobile Action Role Playing Game developed at Arm.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

timeout:

type: 'integer'

The amount of time the game should run for

default: 126

auto_demo:

type: 'boolean'

Start the demo automatically

default: False

show_fps:

type: 'boolean'

Show the FPS count window in-game

default: False

adpf:

type: 'boolean'

Enable ADPF

default: True

adpf_auto:

type: 'boolean'

Enable automatic ADPF mode

default: True

adpf_logging:

type: 'boolean'

Enable ADPF logging

default: False

verbose_log:

type: 'boolean'

Emit reported stats as debug logs

default: False

adpf_interventions:

type: 'boolean'

Enable ADPF interventions

default: True

target_vsyncs:

type: 'integer'

the number of vsyncs to target to a frame (1 = current display rate)

default: 1

target_framerate:

type: 'integer'

Target framerate for the application

fps_report_file:

type: 'str'

File name that the ADPF FPS report should use.

fixed_time_step:

type: 'float'

Time, in seconds, that should be used in advancing the simulation

exoplayer 

Android ExoPlayer

ExoPlayer is the basic video player library that is used by the YouTube android app. The aim of this workload is to test a proxy for YouTube performance on targets where running the real YouTube app is not possible due its dependencies.

ExoPlayer sources: https://github.com/google/ExoPlayer

The ‘demo’ application is used by this workload. It can easily be built by loading the ExoPlayer sources into Android Studio.

Version r2.4.0 built from commit d979469 is known to work

Produces a metric ‘exoplayer_dropped_frames’ - this is the count of frames that Exoplayer itself reports as dropped. This is not the same thing as the dropped frames reported by gfxinfo.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

allowed values: '2.4', '2.5', '2.6'

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

duration:

type: 'integer'

Playback duration of the video file. This becomes the duration of the workload. If provided must be shorter than the length of the media.

default: 20

format:

type: 'str'

Specifies which format video file to play. Default is mov_720p

allowed values: 'mp4_1080p', 'mov_720p', 'mov_480p', 'ogg_128kbps'

filename:

type: 'str'

The name of the video file to play. This can be either a path to the file anywhere on your file system, or it could be just a name, in which case, the workload will look for it in /home/docs/.workload_automation/dependencies/exoplayer Note: either format or filename should be specified, but not both!

force_dependency_push:

type: 'boolean'

If true, video will always be pushed to device, regardless of whether the file is already on the device. Default is False.

default: False

landscape:

type: 'boolean'

Configure the screen in landscape mode, otherwise ensure portrait orientation by default. Default is False.

default: False

geekbench 

Geekbench provides a comprehensive set of benchmarks engineered to quickly and accurately measure processor and memory performance.

http://www.primatelabs.com/geekbench/ From the website: Designed to make benchmarks easy to run and easy to understand, Geekbench takes the guesswork out of producing robust and reliable benchmark results. Geekbench scores are calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are better, with double the score indicating double the performance.

The benchmarks fall into one of four categories:

integer performance.
floating point performance.
memory performance.
stream performance.

Geekbench benchmarks: http://www.primatelabs.com/geekbench/doc/benchmarks.html Geekbench scoring methedology: http://support.primatelabs.com/kb/geekbench/interpreting-geekbench-scores

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

Specifies which version of the workload should be run.

allowed values: '6', '5', '4.4.2', '4.4.0', '4.3.4', '4.3.2', '4.3.1', '4.2.0', '4.0.1', '3.4.1', '3.0.0', '2'

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

loops:

type: 'integer'

Specfies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.

aliases: 'times'

default: 1

timeout:

type: 'integer'

Timeout for a single iteration of the benchmark. This value is multiplied by times to calculate the overall run timeout.

default: 3600

disable_update_result:

type: 'boolean'

If True the results file will not be pulled from the targets /data/data/com.primatelabs.geekbench folder. This allows the workload to be run on unrooted targets and the results extracted manually later.

default: False

geekbench-corporate 

Geekbench provides a comprehensive set of benchmarks engineered to quickly and accurately measure processor and memory performance.

http://www.primatelabs.com/geekbench/ From the website: Designed to make benchmarks easy to run and easy to understand, Geekbench takes the guesswork out of producing robust and reliable benchmark results. Geekbench scores are calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are better, with double the score indicating double the performance.

The benchmarks fall into one of four categories:

integer performance.
floating point performance.
memory performance.
stream performance.

Geekbench benchmarks: http://www.primatelabs.com/geekbench/doc/benchmarks.html Geekbench scoring methedology: http://support.primatelabs.com/kb/geekbench/interpreting-geekbench-scores

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

Specifies which version of the workload should be run.

allowed values: '5.0.3', '5.0.1', '4.1.0', '4.3.4', '5.0.0'

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

loops:

type: 'integer'

Specfies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.

aliases: 'times'

default: 1

timeout:

type: 'integer'

Timeout for a single iteration of the benchmark. This value is multiplied by times to calculate the overall run timeout.

default: 3600

disable_update_result:

type: 'boolean'

If True the results file will not be pulled from the targets /data/data/com.primatelabs.geekbench folder. This allows the workload to be run on unrooted targets and the results extracted manually later.

default: False

gfxbench 

Execute a subset of graphical performance benchmarks

Test description: 1. Open the gfxbench application 2. Execute Car Chase, Manhattan and Tessellation benchmarks

Note: Some of the default tests are unavailable on devices running: with a smaller resolution than 1080p.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

timeout:

type: 'integer'

Timeout for an iteration of the benchmark.

default: 3600

tests:

type: 'list_or_string'

List of tests to be executed.

default: ['Car Chase', '1080p Car Chase Offscreen', 'Manhattan 3.1', '1080p Manhattan 3.1 Offscreen', '1440p Manhattan 3.1.1 Offscreen', 'Tessellation', '1080p Tessellation Offscreen']

gfxbench-corporate 

Execute a subset of graphical performance benchmarks

Test description: 1. Open the gfxbench application 2. Execute Car Chase, Manhattan and Tessellation benchmarks

Note: Some of the default tests are unavailable on devices running: with a smaller resolution than 1080p.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

timeout:

type: 'integer'

Timeout for an iteration of the benchmark.

default: 3600

tests:

type: 'list_or_string'

List of tests to be executed.

default: ['Car Chase', '1080p Car Chase Offscreen', 'Manhattan 3.1', '1080p Manhattan 3.1 Offscreen', '1440p Manhattan 3.1.1 Offscreen', 'Tessellation', '1080p Tessellation Offscreen']

glbenchmark 

Measures the graphics performance of Android devices by testing the underlying OpenGL (ES) implementation.

http://gfxbench.com/about-gfxbench.jsp

From the website:

The benchmark includes console-quality high-level 3D animations (T-Rex HD and Egypt HD) and low-level graphics measurements.

With high vertex count and complex effects such as motion blur, parallax mapping and particle systems, the engine of GFXBench stresses GPUs in order provide users a realistic feedback on their device.

aliases

glbench

egypt: use_case=``’egypt’``
t-rex: use_case=``’t-rex’``
egypt_onscreen: use_case=``’egypt’, type=’onscreen’``
t-rex_onscreen: use_case=``’t-rex’, type=’onscreen’``
egypt_offscreen: use_case=``’egypt’, type=’offscreen’``
t-rex_offscreen: use_case=``’t-rex’, type=’offscreen’``

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

Specifies which version of the benchmark to run (different versions support different use cases).

allowed values: '2.7', '2.5'

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

use_case:

type: 'str'

Specifies which usecase to run, as listed in the benchmark menu; e.g. 'GLBenchmark 2.5 Egypt HD'. For convenience, two aliases are provided for the most common use cases: 'egypt' and 't-rex'. These could be use instead of the full use case title. For version '2.7' it defaults to 't-rex', for version '2.5' it defaults to 'egypt-classic'.

type:

type: 'str'

Specifies which type of the use case to run, as listed in the benchmarks menu (small text underneath the use case name); e.g. 'C24Z16 Onscreen Auto'. For convenience, two aliases are provided for the most common types: 'onscreen' and 'offscreen'. These may be used instead of full type names.

default: 'onscreen'

timeout:

type: 'integer'

Specifies how long, in seconds, UI automation will wait for results screen to appear before assuming something went wrong.

default: 200

gmail 

A workload to perform standard productivity tasks within Gmail. The workload carries out various tasks, such as creating new emails, attaching images and sending them.

Test description: 1. Open Gmail application 2. Click to create New mail 3. Attach an image from the local images folder to the email 4. Enter recipient details in the To field 5. Enter text in the Subject field 6. Enter text in the Compose field 7. Click the Send mail button

To run the workload in offline mode, a ‘mailstore.tar’ file is required. In order to generate such a file, Gmail should first be operated from an Internet-connected environment. After this, the relevant database files can be found in the ‘/data/data/com.google.android.gm/databases’ directory. These files can then be archived to produce a tarball using a command such as tar -cvf mailstore.tar -C /path/to/databases .. The result should then be placed in the ‘~/.workload_automation/dependencies/gmail/’ directory on your local machine, creating this if it does not already exist.

Known working APK version: 2023.04.02.523594694.Release

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

recipient:

type: 'str'

The email address of the recipient. Setting a void address will stop any mesage failures clogging up your device inbox

default: 'wa-devnull@mailinator.com'

test_image:

type: 'str'

An image to be copied onto the device that will be attached to the email

default: 'uxperf_1600x1200.jpg'

offline_mode:

type: 'boolean'

If set to True, the workload will execute in offline mode. This mode requires root and makes use of a tarball of email database files ‘mailstore.tar’ for the email account to be used. This file is extracted directly to the application’s ‘databases’ directory at ‘/data/data/com.google.android.gm/databases’.

default: False

googlemaps 

A workload to perform standard navigation tasks with Google Maps. This workload searches for known locations, pans and zooms around the map, and follows driving directions along a route.

To run the workload in offline mode, databases.tar and files.tar archives are required. In order to generate these files, Google Maps should first be operated from an Internet-connected environment, and a region around Cambridge, England should be downloaded for offline use. This region must include the landmarks used in the UIAutomator program, which include Cambridge train station and Corpus Christi college.

Following this, the files of interest can be found in the databases and files subdirectories of the /data/data/com.google.android.apps.maps/ directory. The contents of these subdirectories can be archived into tarballs using commands such as tar -cvf databases.tar -C /path/to/databases .. These databases.tar and files.tar archives should then be placed in the ~/.workload_automation/dependencies/googlemaps directory on your local machine, creating this if it does not already exist.

Known working APK version: 10.19.1

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

offline_mode:

type: 'boolean'

If set to True, the workload will execute in offline mode. This mode requires root and makes use of a tarball of database files databases.tar and a tarball of auxiliary files files.tar. These tarballs are extracted directly to the application’s databases and files directories respectively in /data/data/com.google.android.apps.maps/.

default: False

googlephotos 

A workload to perform standard productivity tasks with Google Photos. The workload carries out various tasks, such as browsing images, performing zooms, and post-processing the image.

Test description:

Four images are copied to the target
The application is started in offline access mode
Gestures are performed to pinch zoom in and out of the selected image
The colour of a selected image is edited by selecting the colour menu, incrementing the colour, resetting the colour and decrementing the colour using the seek bar.
A crop test is performed on a selected image. UiAutomator does not allow the selection of the crop markers so the image is tilted positively, reset and then tilted negatively to get a similar cropping effect.
A rotate test is performed on a selected image, rotating anticlockwise 90 degrees, 180 degrees and 270 degrees.

Known working APK version: 4.53.0.316914374

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

test_images:

type: 'list_of_strs'

A list of four JPEG and/or PNG files to be pushed to the target. Absolute file paths may be used but tilde expansion must be escaped.

constraint: len(unique(value)) == 4

default: ['uxperf_1200x1600.png', 'uxperf_1600x1200.jpg', 'uxperf_2448x3264.png', 'uxperf_3264x2448.jpg']

googleplaybooks 

A workload to perform standard productivity tasks with googleplaybooks. This workload performs various tasks, such as searching for a book title online, browsing through a book, adding and removing notes, word searching, and querying information about the book.

Test description: 1. Open Google Play Books application 2. Dismisses sync operation (if applicable) 3. Searches for a book title 4. Adds books to library if not already present 5. Opens ‘My Library’ contents 6. Opens selected book 7. Gestures are performed to swipe between pages and pinch zoom in and out of a page 8. Selects a specified chapter based on page number from the navigation view 9. Selects a word in the centre of screen and adds a test note to the page 10. Removes the test note from the page (clean up) 11. Searches for the number of occurrences of a common word throughout the book 12. Switches page styles from ‘Day’ to ‘Night’ to ‘Sepia’ and back to ‘Day’ 13. Uses the ‘About this book’ facility on the currently selected book

NOTE: This workload requires a network connection (ideally, wifi) to run,: a Google account to be setup on the device, and payment details for the account. Free books require payment details to have been setup otherwise it fails. Tip: Install the ‘Google Opinion Rewards’ app to bypass the need to enter valid card/bank detail.

Known working APK version: 3.15.5

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

search_book_title:

type: 'str'

The book title to search for within Google Play Books archive. The book must either be already in the account’s library, or free to purchase.

default: 'Nikola Tesla: Imagination and the Man That Invented the 20th Century'

library_book_title:

type: 'str'

The book title to search for within My Library. The Library name can differ (usually shorter) to the Store name. If left blank, the search_book_title will be used.

default: 'Nikola Tesla'

select_chapter_page_number:

type: 'integer'

The Page Number to search for within a selected book’s Chapter list. Note: Accepts integers only.

default: 4

search_word:

type: 'str'

The word to search for within a selected book. Note: Accepts single words only.

default: 'the'

account:

type: 'str'

If you are running this workload on a device which has more than one Google account setup, then this parameter is used to select which account to select when prompted. The account requires the book to have already been purchased or payment details already associated with the account. If omitted, the first account in the list will be selected if prompted.

googleslides 

A workload to perform standard productivity tasks with Google Slides. The workload carries out various tasks, such as creating a new presentation, adding text, images, and shapes, as well as basic editing and playing a slideshow. This workload should be able to run without a network connection.

There are two main scenarios:

create test: a presentation is created in-app and some editing done on it,
load test: a pre-existing PowerPoint file is copied onto the device for testing.

— create — Create a new file in the application and perform basic editing on it. This test also requires an image file specified by the param test_image to be copied onto the device.

Test description:

Start the app and skip the welcome screen. Dismiss the work offline banner if present.
Go to the app settings page and enables PowerPoint compatibility mode. This allows PowerPoint files to be created inside Google Slides.
Create a new PowerPoint presentation in the app (PPT compatibility mode) with a title slide and save it to device storage.
Insert another slide and to it insert the pushed image by picking it from the gallery.
Insert a final slide and add a shape to it. Resize and drag the shape to modify it.
Finally, navigate back to the documents list.

— load — Copy a PowerPoint presentation onto the device to test slide navigation. The PowerPoint file to be copied is given by test_file.

Test description:

From the documents list (following the create test), open the specified PowerPoint by navigating into device storage and wait for it to be loaded.
A navigation test is performed while the file is in editing mode (i.e. not slideshow). swiping forward to the next slide until slide_count swipes are performed.
While still in editing mode, the same action is done in the reverse direction back to the first slide.
Enter presentation mode by selecting to play the slideshow.
Swipe forward to play the slideshow, for a maximum number of slide_count swipes.
Finally, repeat the previous step in the reverse direction while still in presentation mode, navigating back to the first slide.

NOTE: There are known issues with the reliability of this workload on some targets. It MAY NOT ALWAYS WORK on your device. If you do run into problems, it might help to set do_text_entry parameter to False.

Known working APK version: 1.20.442.04.40

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

test_image:

type: 'str'

An image to be copied onto the device that will be embedded in the PowerPoint file as part of the test.

default: 'uxperf_1600x1200.jpg'

test_file:

type: 'str'

If specified, the workload will copy the PowerPoint file to be used for testing onto the device. Otherwise, a file will be created inside the app.

default: 'uxperf_test_doc.pptx'

slide_count:

type: 'integer'

Number of slides in aforementioned local file. Determines number of swipe actions when playing slide show.

default: 5

do_text_entry:

type: 'boolean'

If set to True, will attempt to enter text in the first slide as part of the test. Currently seems to be problematic on some devices, most notably Samsung devices.

default: True

hackbench 

Hackbench runs a series of tests for the Linux scheduler.

For details, go to: https://github.com/linux-test-project/ltp/

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

timeout:

type: 'integer'

Expected test duration in seconds.

aliases: 'duration'

default: 30

datasize:

type: 'integer'

Message size in bytes.

default: 100

groups:

type: 'integer'

Number of groups.

default: 10

loops:

type: 'integer'

Number of loops.

default: 100

fds:

type: 'integer'

Number of file descriptors.

default: 40

extra_params:

type: 'str'

Extra parameters to pass in. See the hackbench man page or type hackbench –help for list of options.

default: ''

homescreen 

A workload that goes to the home screen and idles for the the specified duration.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

duration:

type: 'integer'

Specifies the duration, in seconds, of this workload.

default: 20

hwuitest 

Tests UI rendering latency on Android devices.

The binary for this workload is built as part of AOSP’s frameworks/base/libs/hwui component.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

test:

type: 'caseless_string'

The test to run:

'shadowgrid': creates a grid of rounded rects that cast shadows, high CPU & GPU load
'rectgrid': creates a grid of 1x1 rects
'oval': draws 1 oval

allowed values: 'shadowgrid', 'rectgrid', 'oval'

default: 'shadowgrid'

loops:

type: 'integer'

The number of test iterations.

default: 3

frames:

type: 'integer'

The number of frames to run the test over.

default: 150

idle 

Do nothing for the specified duration.

On android devices, this may optionally stop the Android run time, if stop_android is set to True.

Note

This workload requires the device to be rooted.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

duration:

type: 'integer'

Specifies the duration, in seconds, of this workload.

default: 20

screen_off:

type: 'boolean'

Ensure that the screen is off before idling.

Note

Make sure screen lock is disabled on the target!

default: False

stop_android:

type: 'boolean'

Specifies whether the Android run time should be stopped. (Can be set only for Android devices).

default: False

jankbench 

Internal Google benchmark for evaluating jank on Android.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

test_ids:

type: 'list_or_string'

ID of the jankbench test to be run.

allowed values: 'list_view', 'image_list_view', 'shadow_grid', 'low_hitrate_text', 'high_hitrate_text', 'edit_text', 'overdraw_test'

loops:

type: 'integer'

Specifies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.

aliases: 'reps'

constraint: value > 0

default: 1

pull_results_db:

type: 'boolean'

Secifies whether an sqlite database with detailed results should be pulled from benchmark app’s data. This requires the device to be rooted.

This defaults to True for rooted devices and False otherwise.

timeout:

type: 'integer'

Time out for workload execution. The workload will be killed if it hasn’t completed within this period.

aliases: 'run_timeout'

default: 600

lmbench 

Run a subtest from lmbench, a suite of portable ANSI/C microbenchmarks for UNIX/POSIX. In general, lmbench measures two key features: latency and bandwidth. This workload supports a subset of lmbench tests. lat_mem_rd can be used to measure latencies to memory (including caches). bw_mem can be used to measure bandwidth to/from memory over a range of operations. Further details, and source code are available from:

http://sourceforge.net/projects/lmbench/.

See lmbench/bin/README for license details.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

test:

type: 'str'

Specifies an lmbench test to run.

allowed values: 'lat_mem_rd', 'bw_mem'

default: 'lat_mem_rd'

stride:

type: '<locals'

Stride for lat_mem_rd test. Workload will iterate over one or more integer values.

default: [128]

thrash:

type: 'boolean'

Sets -t flag for lat_mem_rd_test

default: True

size:

type: 'list_or_string'

Data set size for lat_mem_rd bw_mem tests.

default: '4m'

mem_category:

type: 'list_or_string'

List of memory catetories for bw_mem test.

default: ('rd', 'wr', 'cp', 'frd', 'fwr', 'fcp', 'bzero', 'bcopy')

parallelism:

type: 'integer'

Parallelism flag for tests that accept it.

warmup:

type: 'integer'

Warmup flag for tests that accept it.

repetitions:

type: 'integer'

Repetitions flag for tests that accept it.

force_abi:

type: 'str'

Override device abi with this value. Can be used to force arm32 on 64-bit devices.

run_timeout:

type: 'integer'

Timeout for execution of the test.

default: 900

loops:

type: 'integer'

Specifies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween. This parameter is distinct from “repetitions”, as the latter takes place within the benchmark and produces a single result.

constraint: value > 0

default: 1

cpus:

type: 'cpu_mask'

Specifies the CPU mask the benchmark process will be pinned to.

aliases: 'taskset_mask'

default: 0

manual 

Yields control to the user, either for a fixed period or based on user input, to perform custom operations on the device, which workload automation does not know of.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

duration:

type: 'integer'

Control of the devices is yielded for the duration (in seconds) specified. If not specified, user_triggered is assumed.

user_triggered:

type: 'boolean'

If True, WA will wait for user input after starting the workload; otherwise fixed duration is expected. Defaults to True if duration is not specified, and False otherwise.

view:

type: 'str'

Specifies the View of the workload. This enables instruments that require a View to be specified, such as the fps instrument. This is required for using “SurfaceFlinger” to collect FPS statistics and is primarily used on devices pre API level 23.

default: 'SurfaceView'

package:

type: 'str'

Specifies the package name of the workload. This enables instruments that require a Package to be specified, such as the fps instrument. This allows for “gfxinfo” to be used and is the preferred method of collection for FPS statistics on devices API level 23+.

meabo 

A multi-phased multi-purpose micro-benchmark. The micro-benchmark is composed of 10 phases that perform various generic calculations (from memory to compute intensive).

It is a highly configurable tool which can be used for energy efficiency studies, ARM big.LITTLE Linux scheduler analysis and DVFS studies. It can be used for other benchmarking as well.

All floating-point calculations are double-precision.

Phase 1: Floating-point & integer computations with good data locality
Phase 2: Vector multiplication & addition, 1 level of indirection in 1
source vector
Phase 3: Vector scalar addition and reductions
Phase 4: Vector addition
Phase 5: Vector addition, 1 level of indirection in both source vectors
Phase 6: Sparse matrix-vector multiplication
Phase 7: Linked-list traversal
Phase 8: Electrostatic force calculations
Phase 9: Palindrome calculations
Phase 10: Random memory accesses

For more details and benchmark source, see:

https://github.com/ARM-software/meabo

Note

current implementation of automation relies on the executable to be either statically linked or for all necessary depencies to be installed on the target.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

array_size:

type: 'integer'

Size of arrays used in Phases 1, 2, 3, 4 and 5.

constraint: value > 0

default: 1048576

num_rows:

type: 'integer'

Number of rows for the sparse matrix used in Phase 6.

aliases: 'nrow'

constraint: value > 0

default: 16384

num_cols:

type: 'integer'

Number of columns for the sparse matrix used in Phase 6.

aliases: 'ncol'

constraint: value > 0

default: 16384

loops:

type: 'integer'

Number of iterations that core loop is executed.

aliases: 'num_iterations'

constraint: value > 0

default: 1000

block_size:

type: 'integer'

Block size used in Phase 1.

constraint: value > 0

default: 8

num_cpus:

type: 'integer'

Number of total CPUs that the application can bind threads to.

constraint: value > 0

default: 6

per_phase_cpu_ids:

type: 'list_of_ints'

Sets which cores each phase is run on.

constraint: all(v >= -1 for v in value)

default: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1]

num_hwcntrs:

type: 'integer'

Only available when using PAPI. Controls how many hardware counters PAPI will get access to.

constraint: value >= 0

default: 7

run_phases:

type: 'list_of_ints'

Controls which phases to run.

constraint: all(0 < v <= 10 for v in value)

default: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

threads:

type: 'integer'

Controls how many threads the application will be using.

aliases: 'num_threads'

constraint: value >= 0

default: 0

bind_to_cpu_set:

type: 'integer'

Controls whether threads will be bound to a core set, or each individual thread will be bound to a specific core within the core set.

constraint: 0 <= value <= 1

default: 1

llist_size:

type: 'integer'

Size of the linked list available for each thread.

constraint: value > 0

default: 16777216

num_particles:

type: 'integer'

Number of particles used in Phase 8.

constraint: value > 0

default: 1048576

num_palindromes:

type: 'integer'

Number of palindromes used in Phase 9.

constraint: value > 0

default: 1024

num_randomloc:

type: 'integer'

Number of random memory locations accessed in Phase 10.

constraint: value > 0

default: 2097152

timeout:

type: 'integer'

Timeout for execution of the test.

aliases: 'run_timeout'

constraint: value > 0

default: 2700

memcpy 

Runs memcpy in a loop.

This will run memcpy in a loop for a specified number of times on a buffer of a specified size. Additionally, the affinity of the test can be set to one or more specific cores.

This workload is single-threaded. It generates no scores or metrics by itself.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

buffer_size:

type: 'integer'

Specifies the size, in bytes, of the buffer to be copied.

default: 5242880

loops:

type: 'integer'

Specfies the number of iterations that will be performed.

aliases: 'iterations'

default: 1000

cpus:

type: 'cpu_mask'

The cpus for which the affinity of the test process should be set, specified as a mask, as a list of cpus or a sysfs-style string. If not specified, all available cores will be used.

default: 0

mongoperf 

A utility to check disk I/O performance independently of MongoDB.

It times tests of random disk I/O and presents the results. You can use mongoperf for any case apart from MongoDB. The mmf true mode is completely generic.

Note

mongoperf seems to ramp up threads in powers of two over a period of tens of seconds (there doesn’t appear to be a way to change that). Bear this in mind when setting the duration.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

duration:

type: 'integer'

Duration of of the workload.

default: 300

threads:

type: 'integer'

Defines the number of threads mongoperf will use in the test. To saturate the system storage system you will need multiple threads.

default: 16

file_size_mb:

type: 'integer'

Test file size in MB.

default: 1

sleep_micros:

type: 'integer'

mongoperf will pause for this number of microseconds divided by the the number of threads between each operation.

default: 0

mmf:

type: 'boolean'

When True, use memory mapped files for the tests. Generally:

when mmf is False, mongoperf tests direct, physical, I/O, without caching. Use a large file size to test heavy random I/O load and to avoid I/O coalescing.
when mmf is True, mongoperf runs tests of the caching system, and can use normal file system cache. Use mmf in this mode to test file system cache behavior with memory mapped files.

default: True

read:

type: 'boolean'

When True, perform reads as part of the test. Either read or write must be True.

aliases: 'r'

default: True

write:

type: 'boolean'

When True, perform writes as part of the test. Either read or write must be True.

aliases: 'w'

default: True

rec_size_kb:

type: 'integer'

The size of each write operation

default: 4

sync_delay:

type: 'integer'

Seconds between disk flushes. Only use this if mmf is set to True.

default: 0

motionmark 

A workload to execute the motionmark web based benchmark

MotionMark is a graphics benchmark that measures a browser capability to animate complex scenes at a target frame rate

Test description: 1. Open browser application 2. Navigate to the motionmark website - http://browserbench.org/MotionMark/ 3. Execute the benchmark

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

openssl 

Benchmark Openssl algorithms using Openssl’s speed command.

The command tests how long it takes to perfrom typical SSL operations using a range of supported algorithms and ciphers.

By defalt, this workload will use openssl installed on the target, however it is possible to provide an alternative binary as a workload resource.

aliases

ossl-aes-128-cbc: algorithm=``’aes-128-cbc’``
ossl-aes-192-cbc: algorithm=``’aes-192-cbc’``
ossl-aes-256-cbc: algorithm=``’aes-256-cbc’``
ossl-aes-128-gcm: algorithm=``’aes-128-gcm’``
ossl-aes-192-gcm: algorithm=``’aes-192-gcm’``
ossl-aes-256-gcm: algorithm=``’aes-256-gcm’``
ossl-sha1: algorithm=``’sha1’``
ossl-sha256: algorithm=``’sha256’``
ossl-sha384: algorithm=``’sha384’``
ossl-sha512: algorithm=``’sha512’``
ossl-rsa: algorithm=``’rsa’``
ossl-dsa: algorithm=``’dsa’``
ossl-ecdh: algorithm=``’ecdh’``
ossl-ecdsa: algorithm=``’ecdsa’``

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

algorithm:

type: 'str'

Algorithm to benchmark.

allowed values: 'aes-128-cbc', 'aes-192-cbc', 'aes-256-cbc', 'aes-128-gcm', 'aes-192-gcm', 'aes-256-gcm', 'sha1', 'sha256', 'sha384', 'sha512', 'rsa', 'dsa', 'ecdh', 'ecdsa'

default: 'aes-256-cbc'

threads:

type: 'integer'

The number of threads to use

default: 1

use_system_binary:

type: 'boolean'

If True, the system Openssl binary will be used. Otherwise, use the binary provided in the workload resources.

default: True

pcmark 

A workload to execute the Work x.0 benchmarks within PCMark - https://www.futuremark.com/benchmarks/pcmark-android

Test description: 1. Open PCMark application 2. Swipe right to the Benchmarks screen 3. Select the Work x.0 benchmark 4. If needed, install the Work x.0 benchmark (requires an internet connection) 5. Execute the Work x.0 benchmark

Known working APK versions: 3.0.4061, 2.0.3716

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

Specifies which version of the workload should be run.

allowed values: '3', '2'

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

recentfling 

Tests UI jank on android devices.

For this workload to work, recentfling.sh and defs.sh must be placed in ~/.workload_automation/dependencies/recentfling/. These can be found in the AOSP Git repository.

To change the apps that are opened at the start of the workload you will need to modify the defs.sh file. You will need to add your app to dfltAppList and then add a variable called {app_name}Activity with the name of the activity to launch (where {add_name} is the name you put into dfltAppList).

You can get a list of activities available on your device by running adb shell pm list packages -f

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

loops:

type: 'integer'

The number of test iterations.

default: 3

start_apps:

type: 'boolean'

If set to False,no apps will be started before flinging through the recent apps list (in which the assumption is there are already recently started apps in the list.

default: True

device_name:

type: 'str'

If set, recentfling will use the fling parameters for this device instead of automatically guessing the device. This can also be used if the device is not supported by recentfling, but its screensize is similar to that of one that is supported.

For possible values, check your recentfling.sh. At the time of writing, valid values are: ‘shamu’, ‘hammerhead’, ‘angler’, ‘ariel’, ‘mtp8996’, ‘bullhead’ or ‘volantis’.

rt-app 

A test application that simulates configurable real-time periodic load.

rt-app is a test application that starts multiple periodic threads in order to simulate a real-time periodic load. It supports SCHED_OTHER, SCHED_FIFO, SCHED_RR as well as the AQuoSA framework and SCHED_DEADLINE.

The load is described using JSON-like config files. Below are a couple of simple examples.

Simple use case which creates a thread that run 1ms then sleep 9ms until the use case is stopped with Ctrl+C:

{
    "tasks" : {
        "thread0" : {
            "loop" : -1,
            "run" :   20000,
            "sleep" : 80000
        }
    },
    "global" : {
        "duration" : 2,
        "calibration" : "CPU0",
        "default_policy" : "SCHED_OTHER",
        "pi_enabled" : false,
        "lock_pages" : false,
        "logdir" : "./",
        "log_basename" : "rt-app1",
        "ftrace" : false,
        "gnuplot" : true,
    }
}

Simple use case with 2 threads that runs for 10 ms and wake up each other until the use case is stopped with Ctrl+C

{
    "tasks" : {
        "thread0" : {
            "loop" : -1,
            "run" :     10000,
            "resume" : "thread1",
            "suspend" : "thread0"
        },
        "thread1" : {
            "loop" : -1,
            "run" :     10000,
            "resume" : "thread0",
            "suspend" : "thread1"
        }
    }
}

Please refer to the existing configs in $WA_ROOT/wa/workloads/rt_app/use_case for more examples.

The upstream version of rt-app is hosted here:

https://github.com/scheduler-tools/rt-app

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If set to True, rt-app binary will be uninstalled from the device at the end of the run.

aliases: 'uninstall_on_exit'

default: False

config:

type: 'str'

Use case configuration file to run with rt-app. This may be either the name of one of the “standard” configurations included with the workload. or a path to a custom JSON file provided by the user. Either way, the “.json” extension is implied and will be added automatically if not specified in the argument.

The following is the list of standard configurations currently included with the workload: camera-long.json, taskset.json, camera-short.json, browser-short.json, mp3-long.json, spreading-tasks.json, video-short.json, mp3-short.json, browser-long.json, video-long.json

default: 'taskset'

duration:

type: 'integer'

Duration of the workload execution in Seconds. If specified, this will override the corresponding parameter in the JSON config.

cpus:

type: 'cpu_mask'

Constrain execution to specific CPUs.

aliases: 'taskset_mask'

default: 0

force_install:

type: 'boolean'

If set to True, rt-app binary will always be deployed to the target device at the beginning of the run, regardless of whether it was already installed there.

default: False

schbench 

Benchmark providing detailed latency distribution statistics for scheduler wakeups.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

runtime:

type: 'integer'

How long to run before exiting (seconds)

aliases: 'timeout'

default: 30

message_threads:

type: 'integer'

number of message threads

default: 2

threads:

type: 'integer'

worker threads per message thread

default: 16

sleeptime:

type: 'integer'

Message thread latency (usec)

default: 30000

cputime:

type: 'integer'

How long to think during loop

default: 30000

auto:

type: 'boolean'

grow thread count until latencies hurt

default: False

pipe:

type: 'integer'

transfer size bytes to simulate a pipe test

default: 0

rps:

type: 'integer'

requests per second mode (count)

default: 0

intervaltime:

type: 'integer'

interval for printing latencies (seconds)

shellscript 

Runs an arbitrary shellscript on the target.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

script_file: (mandatory)

type: 'str'

The path (on the host) to the shell script file. This must be an absolute path (though it may contain ~).

argstring:

type: 'str'

A string that should contain arguments passed to the script.

default: ''

as_root:

type: 'boolean'

Specify whether the script should be run as root.

default: False

timeout:

type: 'integer'

Timeout, in seconds, for the script run time.

default: 60

speedometer 

A workload to execute the speedometer 2.0 web based benchmark. Requires device to be rooted. This workload will only with Android 9+ devices if connected via TCP, or Android 5+ if connected via USB.

Test description:

Host a local copy of the Speedometer website, and make it visible to the device via ADB.
Open chrome via an intent to access the local copy.
Execute the benchmark - the copy has been modified to trigger the start of the benchmark.
The benchmark will write to the browser’s sandboxed local storage to signal the benchmark has completed. This local storage is monitored by this workload.

Known working chrome version 83.0.4103.106

To modify the archived speedometer workload:

Run ‘git clone https://github.com/WebKit/webkit’
Copy PerformanceTests/Speedometer to a directory called document_root, renaming Speedometer to Speedometer2.0
Modify document_root/Speedometer2.0/index.html:

3a. Remove the ‘defer’ attribute from the <script> tags within the <head> section. 3b. Add ‘<script>startTest();</script>’ to the very end of the <body> section.

Modify document_root/Speedometer2.0/resources/main.js:

4a. Add the listed code after this line:

document.getElementById(‘result-number’).textContent = results.formattedMean;

Code to add:

if (location.search.length > 1) {
    var parts = location.search.substring(1).split('&');
    for (var i = 0; i < parts.length; i++) {
        var keyValue = parts[i].split('=');
        var key = keyValue[0];
        var value = keyValue[1];
        if (key === "reportEndId") {
            window.localStorage.setItem('reportEndId', value);
        }
    }
}

Run ‘tar -cpzf speedometer_archive.tgz document_root’
Copy the tarball into the workloads/speedometer directory
If appropriate, update the commit info in the LICENSE file.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

chrome_package:

type: 'str'

The app package for the browser that will be launched.

allowed values: 'org.chromium.chrome', 'com.android.chrome', 'org.bromite.chromium'

default: 'com.android.chrome'

stress-ng 

Run the stress-ng benchmark.

stress-ng will stress test a computer system in various selectable ways. It was designed to exercise various physical subsystems of a computer as well as the various operating system kernel interfaces.

stress-ng can also measure test throughput rates; this can be useful to observe performance changes across different operating system releases or types of hardware. However, it has never been intended to be used as a precise benchmark test suite, so do NOT use it in this manner.

The official website for stress-ng is at:: http://kernel.ubuntu.com/~cking/stress-ng/
Source code are available from:: http://kernel.ubuntu.com/git/cking/stress-ng.git/

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

stressor:

type: 'str'

Stress test case name. The cases listed in allowed values come from the stable release version 0.01.32. The binary included here compiled from dev version 0.06.01. Refer to man page for the definition of each stressor.

allowed values: 'cpu', 'io', 'fork', 'switch', 'vm', 'pipe', 'yield', 'hdd', 'cache', 'sock', 'fallocate', 'flock', 'affinity', 'timer', 'dentry', 'urandom', 'sem', 'open', 'sigq', 'poll'

default: 'cpu'

extra_args:

type: 'str'

Extra arguments to pass to the workload.

Please note that these are not checked for validity.

default: ''

threads:

type: 'integer'

The number of workers to run. Specifying a negative or zero value will select the number of online processors.

default: 0

duration:

type: 'integer'

Timeout for test execution in seconds

default: 60

sysbench 

A modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load.

The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.

Features of SysBench

file I/O performance

scheduler performance

memory allocation and transfer speed

POSIX threads implementation performance

database server performance

See: https://github.com/akopytov/sysbench

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, executables that are installed to the device as part of the workload will be uninstalled again.

default: True

timeout:

type: 'integer'

timeout for workload execution (adjust from default if running on a slow target and/or specifying a large value for max_requests

default: 300

test:

type: 'str'

sysbench test to run

allowed values: 'fileio', 'cpu', 'memory', 'threads', 'mutex'

default: 'cpu'

threads:

type: 'integer'

The number of threads sysbench will launch.

aliases: 'num_threads'

default: 8

max_requests:

type: 'integer'

The limit for the total number of requests.

max_time:

type: 'integer'

The limit for the total execution time. If neither this nor max_requests is specified, this will default to 30 seconds.

file_test_mode:

type: 'str'

File test mode to use. This should only be specified if test is "fileio"; if that is the case and file_test_mode is not specified, it will default to "seqwr" (please see sysbench documentation for explanation of various modes).

allowed values: 'seqwr', 'seqrewr', 'seqrd', 'rndrd', 'rndwr', 'rndrw'

cmd_params:

type: 'str'

Additional parameters to be passed to sysbench as a single string.

default: ''

cpus:

type: 'cpu_mask'

The processes spawned by sysbench will be pinned to cores as specified by this parameter. Can be provided as a mask, a list of cpus or a sysfs-style string.

aliases: 'taskset_mask'

default: 0

templerun2 

Temple Run 2 game.

Sequel to Temple Run. 3D on-the-rails racer.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

thechase 

The Chase demo showcasing the capabilities of Unity game engine.

This demo, is a static video-like game demo, that demonstrates advanced features of the unity game engine. It loops continuously until terminated.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

duration:

type: 'integer'

Duration, in seconds, note that the demo loops the same (roughly) 60 second sceene until stopped.

default: 70

uibench 

Runs a particular activity of the UIBench workload suite. The suite is provided by Google as a testbench for the Android UI.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

activity:

type: 'str'

The UIBench activity to be run. Each activity corresponds to a test. If this parameter is ignored, the application is launched in its main menu. Please note that the available activities vary between versions of UIBench (which follow AOSP versioning) and the availability of the services under test may depend on the version of the target Android. We recommend using the APK of UIBench corresponding to the Android version, enforced through the version parameter to this workload.

duration:

type: 'integer'

As activities do not finish, this workload will terminate UIBench after the given duration.

default: 10

uibenchjanktests 

Runs a particular test (or list of tests) of the UIBench JankTests test suite. The suite is provided by Google as an automated version of the UIBench testbench for the Android UI. The workload supports running the default set of tests without restarting the app or running an arbitrary set of tests with restarting the app in between each test.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

tests:

type: 'list_or_string'

Tests to be run. Defaults to running every available subtest in alphabetical order. The app will be restarted for each subtest, unlike when using full=True.

aliases: 'test'

full:

type: 'boolean'

Runs the full suite of tests that the app defaults to when no subtests are specified. The actual tests and their order might depend on the version of the app. The subtests will be run back to back without restarting the app in between.

default: False

wait:

type: 'boolean'

Forces am instrument to wait until the instrumentation terminates before terminating itself. The net effect is to keep the shell open until the tests have finished. This flag is not required, but if you do not use it, you will not see the results of your tests.

default: True

raw:

type: 'boolean'

Outputs results in raw format. Use this flag when you want to collect performance measurements, so that they are not formatted as test results. This flag is designed for use with the flag -e perf true.

default: True

instrument_args:

type: 'OrderedDict'

Extra arguments for am instrument.

default:

{

}

no_hidden_api_checks:

type: 'boolean'

Disables restrictions on the use of hidden APIs.

default: False

vellamo 

Android benchmark designed by Qualcomm.

Vellamo began as a mobile web benchmarking tool that today has expanded to include three primary chapters. The Browser Chapter evaluates mobile web browser performance, the Multicore chapter measures the synergy of multiple CPU cores, and the Metal Chapter measures the CPU subsystem performance of mobile processors. Through click-and-go test suites, organized by chapter, Vellamo is designed to evaluate: UX, 3D graphics, and memory read/write and peak bandwidth performance, and much more!

Note: Vellamo v3.0 fails to run on Juno

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

Specify the version of Vellamo to be run. If not specified, the latest available version will be used.

allowed values: '2.0.3', '3.0', '3.2.4'

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

benchmarks:

type: 'list_of_strs'

Specify which benchmark sections of Vellamo to be run. Only valid on version 3.0 and newer. NOTE: Browser benchmark can be problematic and seem to hang,just wait and it will progress after ~5 minutes

allowed values: 'Browser', 'Metal', 'Multi'

default: ['Browser', 'Metal', 'Multi']

browser:

type: 'integer'

Specify which of the installed browsers will be used for the tests. The number refers to the order in which browsers are listed by Vellamo. E.g. 1 will select the first browser listed, 2 – the second, etc. Only valid for version 3.0.

default: 1

youtube 

A workload to perform standard productivity tasks within YouTube.

The workload plays a video from the app, determined by the video_source parameter. While the video is playing, a some common actions are done such as video seeking, pausing playback and navigating the comments section.

Test description: The video_source parameter determines where the video to be played will be found in the app. Possible values are search, home, my_videos, and trending.

-A. search - Goes to the search view, does a search for the given term, and plays the: first video in the results. The parameter search_term must also be provided in the agenda for this to work. This is the default mode.
-B. home - Scrolls down once on the app’s home page to avoid ads (if present, would be: first video), then select and plays the video that appears at the top of the list.
-C. my_videos - Goes to the ‘My Videos’ section of the user’s account page and plays a: video from there. The user must have at least one uploaded video for this to work.
-D. trending - Goes to the ‘Trending Videos’ section of the app, and plays the first: video in the trending videos list.

For the selected video source, the following test steps are performed:

Navigate to the general app settings page to disable autoplay. This improves test stability and predictability by preventing screen transition to load a new video while in the middle of the test.
Select the video from the source specified above, and dismiss any potential embedded advert that may pop-up before the actual video.
Let the video play for a few seconds, pause it, then resume.
Expand the info card that shows video metadata, then collapse it again.
Scroll down to the end of related videos and comments under the info card, and then back up to the start. A maximum of 5 swipe actions is performed in either direction.

Known working APK version: 15.45.32

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

markers_enabled:

type: 'boolean'

If set to True, workloads will insert markers into logs at various points during execution. These markers may be used by other plugins or post-processing scripts to provide measurements or statistics for specific parts of the workload execution.

default: False

video_source:

type: 'str'

Determines where to play the video from. This can either be from the YouTube home, my videos section, trending videos or found in search.

allowed values: 'home', 'my_videos', 'search', 'trending'

default: 'search'

search_term:

type: 'str'

The search term to use when video_source is set to search. Ignored otherwise.

default: 'Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film'

youtube_playback 

Simple Youtube video playback

This triggers a video streaming playback on Youtube. Unlike the more featureful “youtube” workload, this performs no other action that starting the video via an intent and then waiting for a certain amount of playback time. This is therefore only useful when you are confident that the content on the end of the provided URL is stable - that means the video should have no advertisements attached.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

uninstall:

type: 'boolean'

If True, will uninstall workload’s APK as part of teardown.’

default: False

package_name:

type: 'str'

The package name that can be used to specify the workload apk to use.

install_timeout:

type: 'integer'

Timeout for the installation of the apk.

constraint: value > 0

default: 300

version:

type: 'str'

The version of the package to be used.

max_version:

type: 'str'

The maximum version of the package to be used.

min_version:

type: 'str'

The minimum version of the package to be used.

variant:

type: 'str'

The variant of the package to be used.

strict:

type: 'boolean'

Whether to throw an error if the specified package cannot be found on host.

default: False

force_install:

type: 'boolean'

Always re-install the APK, even if matching version is found already installed on the device.

default: False

exact_abi:

type: 'boolean'

If True, workload will check that the APK matches the target device ABI, otherwise any suitable APK found will be used.

default: False

prefer_host_package:

type: 'boolean'

If True then a package found on the host will be preferred if it is a valid version and ABI, if not it will fall back to the version on the target if available. If False then the version on the target is preferred instead.

aliases: 'check_apk'

default: True

view:

type: 'str'

Manually override the ‘View’ of the workload for use with instruments such as the fps instrument. If not specified, a workload dependant ‘View’ will be automatically generated.

video_url:

type: 'str'

URL of video to play

default: 'https://www.youtube.com/watch?v=YE7VzlLtp-4'

duration:

type: 'integer'

Number of seconds of video to play

default: 20