Workloads

adobereader

The Adobe Reader workflow carries out the following typical productivity tasks.

Test description:

1. Open a local file on the device

2. Gestures test:

   2.1. Swipe down across the central 50% of the screen in 200 x 5ms steps 2.2. Swipe up across the central 50% of the screen in 200 x 5ms steps 2.3. Swipe right from the edge of the screen in 50 x 5ms steps 2.4. Swipe left from the edge of the screen in 50 x 5ms steps 2.5. Pinch out 50% in 100 x 5ms steps 2.6. Pinch In 50% in 100 x 5ms steps

3. Search test:

   Search document_name for each string in the search_string_list

4. Close the document

Known working APK version: 16.1

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

document_name : str

The document name to use for the Gesture and Search test.

default: 'uxperf_test_doc.pdf'

search_string_list : list_of_strs

For each string in the list, a document search is performed using the string as the search term. At least one must be provided.

constraint: len(value) > 0

default: ['The quick brown fox jumps over the lazy dog', 'TEST_SEARCH_STRING']

andebench

AndEBench is an industry standard Android benchmark provided by The Embedded Microprocessor Benchmark Consortium (EEMBC).

http://www.eembc.org/andebench/about.php

From the website:

• Initial focus on CPU and Dalvik interpreter performance
• Internal algorithms concentrate on integer operations
• Compares the difference between native and Java performance
• Implements flexible multicore performance analysis
• Results displayed in Iterations per second
• Detailed log file for comprehensive engineering analysis

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

number_of_threads : integer

Number of threads that will be spawned by AndEBench.

single_threaded : boolean

If true, AndEBench will run with a single thread. Note: this must not be specified if number_of_threads has been specified.

native_only : boolean

If true, AndEBench will execute only the native portion of the benchmark.

androbench

Measures the storage performance of an Android device.

Website: http://www.androbench.org/wiki/AndroBench

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

angrybirds

Angry Birds game.

A very popular Android 2D game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

angrybirds_rio

Angry Birds Rio game.

The sequel to the very popular Android 2D game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

anomaly2

Anomaly 2 game demo and benchmark.

Plays three scenes from the game, benchmarking each one. Scores reported are intended to represent overall perceived quality of the game, based not only on raw FPS but also factors like smoothness.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

antutu

AnTuTu Benchmark is an benchmarking tool for Android Mobile Phone/Pad. It can run a full test of a key project, through the “Memory Performance”,”CPU Integer Performance”,”CPU Floating point Performance”,”2D 3D Graphics Performance”,”SD card reading/writing speed”,”Database IO” performance testing, and gives accurate analysis for Andriod smart phones.

http://www.antutulabs.com/AnTuTu-Benchmark

From the website:

AnTuTu Benchmark can support the latest quad-core cpu. In reaching the overall and individual scores of the hardware, AnTuTu Benchmark could judge your phone by the scores of the performance of the hardware. By uploading the scores, Benchmark can view your device in the world rankings, allowing points to let you know the level of hardware performance equipment.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

version : str

Specify the version of AnTuTu to be run. If not specified, the latest available version will be used.

allowed values: '3.3.2', '4.0.3', '5.3.0', '6.0.1'

default: '6.0.1'

times : integer

The number of times the benchmark will be executed in a row (i.e. without going through the full setup/teardown process). Note: this does not work with versions prior to 4.0.3.

default: 1

enable_sd_tests : boolean

If True enables SD card tests in pre version 4 AnTuTu. These tests were know to cause problems on platforms without an SD card. This parameter will be ignored on AnTuTu version 4 and higher.

apklaunch

Installs and runs a .apk file, waits wait_time_seconds, and tests if the app has started successfully.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

apk_file : str (mandatory)

Name to the .apk to run

uninstall_required : boolean

Set to true if the package should be uninstalled

wait_time_seconds : integer

Seconds to wait before testing if the app is still alive

applaunch

This workload launches and measures the launch time of applications for supporting workloads.

Currently supported workloads are the ones that implement ApplaunchInterface. For any workload to support this workload, it should implement the ApplaunchInterface. The corresponding java file of the workload associated with the application being measured is executed during the run. The application that needs to be measured is passed as a parametre workload_name. The parameters required for that workload have to be passed as a dictionary which is captured by the parametre workload_params. This information can be obtained by inspecting the workload details of the specific workload.

The workload allows to run multiple iterations of an application launch in two modes:

1. Launch from background
2. Launch from long-idle

These modes are captured as a parameter applaunch_type.

launch_from_background

Launches an application after the application is sent to background by pressing Home button.

launch_from_long-idle

Launches an application after killing an application process and clearing all the caches.

Test Description:

• During the initialization and setup, the application being launched is launched for the first time. The jar file of the workload of the application is moved to device at the location workdir which further implements the methods needed to measure the application launch time.

• Run phase calls the UiAutomator of the applaunch which runs in two subphases.

  1. Applaunch Setup Run:

     During this phase, welcome screens and dialogues during the first launch of the instrumented application are cleared.

  2. Applaunch Metric Run:

     During this phase, the application is launched multiple times determined by the iteration number specified by the parametre applaunch_iterations. Each of these iterations are instrumented to capture the launch time taken and the values are recorded as UXPERF marker values in logfile.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

workload_name : str

Name of the uxperf workload to launch

default: 'gmail'

workload_params : dict

parameters of the uxperf workload whose application launch time is measured

applaunch_type : str

Choose launch_from_long-idle for measuring launch time from long-idle. These two types are described in the class description.

allowed values: 'launch_from_background', 'launch_from_long-idle'

default: 'launch_from_background'

applaunch_iterations : integer

Number of iterations of the application launch

default: 1

report_results : boolean

Choose to report results of the application launch time.

default: True

appshare

Workload to test how responsive a device is when context switching between application tasks. It combines workflows from googlephotos, gmail and skype.

** Setup ** Credentials for the user account used to log into the Skype app have to be provided in the agenda, as well as the display name of the contact to call.

For reliable testing, this workload requires a good and stable internet connection, preferably on Wi-Fi.

Although this workload attempts to be network independent it requires a network connection (ideally, wifi) to run. This is because the welcome screen UI is dependent on an existing connection.

Test description:

1. GooglePhotos is started in offline access mode

   1.1. The welcome screen is dismissed 1.2. Any promotion popup is dismissed 1.3. The provided test_image is selected and displayed

2. The image is then shared across apps to Gmail

   2.1. The first run dialogue is dismissed 2.2. Enter recipient details in the To field 2.3. Enter text in the Subject field 2.4. Enter text in the Body field 2.5. Click the Send mail button

3. Return to Googlephotos and login to Skype via share action

4. Return to Googlephotos and share the test_image with Skype

   4.1. Search for the skype_contact_name from the Contacts list 4.2. Dismiss any update popup that appears 4.3. The image is posted in the Chat

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

test_image : str

An image to be copied onto the device that will be shared across multiple apps

default: 'uxperf_1600x1200.jpg'

email_recipient : str

The email address of the recipient to recieve the shared image

default: 'wa-devnull@mailinator.com'

skype_login_name : str (mandatory)

Account to use when logging into skype from which to share the image

skype_login_pass : str (mandatory)

Password associated with the skype account

skype_contact_name : str

This is the contact display name as it appears in the people list

default: 'Echo / Sound Test Service'

audio

Audio workload plays an MP3 file using the built-in music player. By default, it plays Canon_in_D_Pieano.mp3 for 30 seconds.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

duration : integer

The duration the music will play for in seconds.

default: 30

audio_file : str

The (on-host) path to the audio file to be played.

Note

If the default file is not present locally, it will be downloaded.

default: '~/.workload_automation/dependencies/Canon_in_D_Piano.mp3'

perform_cleanup : boolean

If True, workload files on the device will be deleted after execution.

clear_file_cache : boolean

Clear the the file cache on the target device prior to running the workload.

default: True

autotest

Executes tests from ChromeOS autotest suite

Note

This workload must be run inside a CromeOS SDK chroot.

See: https://www.chromium.org/chromium-os/testing/power-testing

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

test : str (mandatory)

The test to be run

test_that_args : arguments

Extra arguments to be passed to test_that_invocation.

run_timeout : integer

Timeout, in seconds, for the test execution.

default: 1800

bbench

BBench workload opens the built-in browser and navigates to, and scrolls through, some preloaded web pages and ends the workload by trying to connect to a local server it runs after it starts. It can also play the workload while it plays an audio file in the background.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

with_audio : boolean

Specifies whether an MP3 should be played in the background during workload execution.

server_timeout : integer

Specifies the timeout (in seconds) before the server is stopped.

default: 300

force_dependency_push : boolean

Specifies whether to push dependency files to the device to the device if they are already on it.

audio_file : str

The (on-host) path to the audio file to be played. This is only used if with_audio is True.

default: '~/.workload_automation/dependencies/Canon_in_D_Piano.mp3'

perform_cleanup : boolean

If True, workload files on the device will be deleted after execution.

clear_file_cache : boolean

Clear the the file cache on the target device prior to running the workload.

default: True

browser_package : str

Specifies the package name of the device’s browser app.

default: 'com.android.browser'

browser_activity : str

Specifies the startup activity name of the device’s browser app.

default: '.BrowserActivity'

benchmarkpi

Measures the time the target device takes to run and complete the Pi calculation algorithm.

http://androidbenchmark.com/howitworks.php

from the website:

The whole idea behind this application is to use the same Pi calculation algorithm on every Android Device and check how fast that proccess is. Better calculation times, conclude to faster Android devices. This way you can also check how lightweight your custom made Android build is. Or not.

As Pi is an irrational number, Benchmark Pi does not calculate the actual Pi number, but an approximation near the first digits of Pi over the same calculation circles the algorithms needs.

So, the number you are getting in miliseconds is the time your mobile device takes to run and complete the Pi calculation algorithm resulting in a approximation of the first Pi digits.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

blogbench

Blogbench is a portable filesystem benchmark that tries to reproduce the load of a real-world busy file server.

Blogbench stresses the filesystem with multiple threads performing random reads, writes and rewrites in order to get a realistic idea of the scalability and the concurrency a system can handle.

Source code are available from:

https://download.pureftpd.org/pub/blogbench/

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

iterations : integer

The number of iterations to run

default: 30

caffeinemark

CaffeineMark is a series of tests that measure the speed of Java programs running in various hardware and software configurations.

http://www.benchmarkhq.ru/cm30/info.html

From the website:

CaffeineMark scores roughly correlate with the number of Java instructions executed per second, and do not depend significantly on the the amount of memory in the system or on the speed of a computers disk drives or internet connection.

The following is a brief description of what each test does:

• Sieve: The classic sieve of eratosthenes finds prime numbers.

• Loop: The loop test uses sorting and sequence generation as to measure

  compiler optimization of loops.

• Logic: Tests the speed with which the virtual machine executes

  decision-making instructions.

• Method: The Method test executes recursive function calls to see how

  well the VM handles method calls.

• Float: Simulates a 3D rotation of objects around a point.
• Graphics: Draws random rectangles and lines.
• Image: Draws a sequence of three graphics repeatedly.
• Dialog: Writes a set of values into labels and editboxes on a form.

The overall CaffeineMark score is the geometric mean of the individual scores, i.e., it is the 9th root of the product of all the scores.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

cameracapture

Uses in-built Android camera app to take photos.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

no_of_captures : integer

Number of photos to be taken.

default: 5

time_between_captures : integer

Time, in seconds, between two consecutive camera clicks.

default: 5

camerarecord

Uses in-built Android camera app to record the video for given interval of time.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

recording_time : integer

The video recording time in seconds.

default: 60

recording_mode : str

The video recording mode.

allowed values: 'normal', 'slow_motion'

default: 'normal'

castlebuilder

Castle Builder game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

castlemaster

Castle Master v1.09 game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

cfbench

CF-Bench is (mainly) CPU and memory benchmark tool specifically designed to be able to handle multi-core devices, produce a fairly stable score, and test both native as well managed code performance.

https://play.google.com/store/apps/details?id=eu.chainfire.cfbench&hl=en

From the website:

It tests specific device properties you do not regularly see tested by other benchmarks, and runs in a set timeframe.

It does produce some “final” scores, but as with every benchmark, you should take those with a grain of salt. It is simply not theoretically possible to produce a single number that accurately describes a device’s performance.

Note

This workload relies on the device being rooted

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

citadel

Epic Citadel demo showcasing Unreal Engine 3.

The game has very rich graphics details. The workload only moves around its environment for the specified time.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

duration : integer

Duration, in seconds, of the run (may need to be adjusted for different devices.

default: 60

cyclictest

Measures the amount of time that passes between when a timer expires and when the thread which set the timer actually runs.

Cyclic test works by taking a time snapshot just prior to waiting for a specific time interval (t1), then taking another time snapshot after the timer finishes (t2), then comparing the theoretical wakeup time with the actual wakeup time (t2 -(t1 + sleep_time)). This value is the latency for that timers wakeup.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

clock : str

specify the clock to be used during the test.

allowed values: 'monotonic', 'realtime'

default: 'realtime'

duration : integer

Specify the length for the test to run in seconds.

default: 30

quiet : boolean

Run the tests quiet and print only a summary on exit.

default: True

thread : integer

Set the number of test threads

default: 8

latency : integer

Write the value to /dev/cpu_dma_latency

default: 1000000

extra_parameters : str

Any additional command line parameters to append to the existing parameters above. A list can be found at https://rt.wiki.kernel.org/index.php/Cyclictest or in the help page cyclictest -h

clear_file_cache : boolean

Clear file caches before starting test

default: True

screen_off : boolean

If true it will turn the screen off so that onscreen graphics do not effect the score. This is predominantly for devices without a GPU

default: True

dex2oat

Benchmarks the execution time of dex2oat (a key part of APK installation process).

ART is a new Android runtime in KitKat, which replaces Dalvik VM. ART uses Ahead-Of-Time compilation. It pre-compiles ODEX files used by Dalvik using dex2oat tool as part of APK installation process.

This workload benchmarks the time it take to compile an APK using dex2oat, which has a significant impact on the total APK installation time, and therefore user experience.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

instruction_set : str

Specifies the instruction set to compile for. Only options supported by the target device can be used.

allowed values: 'arm', 'arm64', 'x86', 'x86_64', 'mips'

default: 'arm64'

dhrystone

Runs the Dhrystone benchmark.

Original source from:

http://classes.soe.ucsc.edu/cmpe202/benchmarks/standard/dhrystone.c

This version has been modified to configure duration and the number of threads used.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

duration : integer

The duration, in seconds, for which dhrystone will be executed. Either this or mloops should be specified but not both.

mloops : integer

Millions of loops to run. Either this or duration should be specified, but not both. If neither is specified, this will default to 100

threads : integer

The number of separate dhrystone “threads” that will be forked.

default: 4

delay : integer

The delay, in seconds, between kicking off of dhrystone threads (if threads > 1).

taskset_mask : integer

The processes spawned by the workload will be pinned to cores as specified by this parameter

dungeondefenders

Dungeon Defenders game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

ebizzy

ebizzy is designed to generate a workload resembling common web application server workloads. It is highly threaded, has a large in-memory working set with low locality, and allocates and deallocates memory frequently. When running most efficiently, it will max out the CPU.

ebizzy description taken from the source code at https://github.com/linux-test-project/ltp/tree/master/utils/benchmark/ebizzy-0.3

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

threads : integer

Number of threads to execute.

default: 2

seconds : integer

Number of seconds.

default: 10

chunks : integer

Number of memory chunks to allocate.

default: 10

extra_params : str

Extra parameters to pass in (e.g. -M to disable mmap). See ebizzy -? for full list of options.

facebook

Uses com.facebook.patana apk for facebook workload. This workload does the following activities in facebook

Login to facebook account. Send a message. Check latest notification. Search particular user account and visit his/her facebook account. Find friends. Update the facebook status

Note

This workload starts disableUpdate workload as a part of setup to disable online updates, which helps to tackle problem of uncertain behavier during facebook workload run.]

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

geekbench

Geekbench provides a comprehensive set of benchmarks engineered to quickly and accurately measure processor and memory performance.

http://www.primatelabs.com/geekbench/

From the website:

Designed to make benchmarks easy to run and easy to understand, Geekbench takes the guesswork out of producing robust and reliable benchmark results.

Geekbench scores are calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are better, with double the score indicating double the performance.

The benchmarks fall into one of four categories:

• integer performance.
• floating point performance.
• memory performance.
• stream performance.

Geekbench benchmarks: http://www.primatelabs.com/geekbench/doc/benchmarks.html

Geekbench scoring methedology: http://support.primatelabs.com/kb/geekbench/interpreting-geekbench-scores

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

version : str

Specifies which version of the workload should be run.

allowed values: '2', '3.0.0', '3.4.1', '4.0.1'

default: '4.0.1'

times : integer

Specfies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.

default: 1

timeout : integer

Timeout for a single iteration of the benchmark. This value is multiplied by times to calculate the overall run timeout.

default: 900

disable_update_result : boolean

If True the results file will not be pulled from the devices /data/data/com.primatelabs.geekbench folder. This allows the workload to be run on unrooted devices and the results extracted manually later.

geekbench-corporate

Geekbench provides a comprehensive set of benchmarks engineered to quickly and accurately measure processor and memory performance.

http://www.primatelabs.com/geekbench/

From the website:

Designed to make benchmarks easy to run and easy to understand, Geekbench takes the guesswork out of producing robust and reliable benchmark results.

Geekbench scores are calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are better, with double the score indicating double the performance.

The benchmarks fall into one of four categories:

• integer performance.
• floating point performance.
• memory performance.
• stream performance.

Geekbench benchmarks: http://www.primatelabs.com/geekbench/doc/benchmarks.html

Geekbench scoring methedology: http://support.primatelabs.com/kb/geekbench/interpreting-geekbench-scores

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

version : str

Specifies which version of the workload should be run.

allowed values: '4.1.0'

default: '4.1.0'

times : integer

Specfies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.

default: 1

timeout : integer

Timeout for a single iteration of the benchmark. This value is multiplied by times to calculate the overall run timeout.

default: 900

disable_update_result : boolean

If True the results file will not be pulled from the devices /data/data/com.primatelabs.geekbench folder. This allows the workload to be run on unrooted devices and the results extracted manually later.

glb_corporate

GFXBench GL (a.k.a. GLBench) v3.0 Corporate version.

This is a version of GLBench available through a corporate license (distinct from the version available in Google Play store).

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

times : integer

Specifies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.

constraint: value > 0

default: 1

resolution : str

Explicitly specifies the resultion under which the benchmark will be run. If not specfied, device’s native resoution will used.

allowed values: '720p', '1080p', '720', '1080'

test_id : str

ID of the GFXBench test to be run.

allowed values: 'gl_alu', 'gl_alu_off', 'gl_blending', 'gl_blending_off', 'gl_driver', 'gl_driver_off', 'gl_fill', 'gl_fill_off', 'gl_manhattan', 'gl_manhattan_off', 'gl_trex', 'gl_trex_battery', 'gl_trex_off', 'gl_trex_qmatch', 'gl_trex_qmatch_highp'

default: 'gl_manhattan_off'

run_timeout : integer

Time out for workload execution. The workload will be killed if it hasn’t completed withint this period.

default: 600

glbenchmark

Measures the graphics performance of Android devices by testing the underlying OpenGL (ES) implementation.

http://gfxbench.com/about-gfxbench.jsp

From the website:

The benchmark includes console-quality high-level 3D animations (T-Rex HD and Egypt HD) and low-level graphics measurements.

With high vertex count and complex effects such as motion blur, parallax mapping and particle systems, the engine of GFXBench stresses GPUs in order provide users a realistic feedback on their device.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

version : str

Specifies which version of the benchmark to run (different versions support different use cases).

allowed values: '2.7', '2.5'

default: '2.7'

use_case : str

Specifies which usecase to run, as listed in the benchmark menu; e.g. 'GLBenchmark 2.5 Egypt HD'. For convenience, two aliases are provided for the most common use cases: 'egypt' and 't-rex'. These could be use instead of the full use case title. For version '2.7' it defaults to 't-rex', for version '2.5' it defaults to 'egypt-classic'.

variant : str

Specifies which variant of the use case to run, as listed in the benchmarks menu (small text underneath the use case name); e.g. 'C24Z16 Onscreen Auto'. For convenience, two aliases are provided for the most common variants: 'onscreen' and 'offscreen'. These may be used instead of full variant names.

default: 'onscreen'

times : integer

Specfies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween.

default: 1

timeout : integer

Specifies how long, in seconds, UI automation will wait for results screen to appear before assuming something went wrong.

default: 200

gmail

A workload to perform standard productivity tasks within Gmail. The workload carries out various tasks, such as creating new emails, attaching images and sending them.

Test description: 1. Open Gmail application 2. Click to create New mail 3. Attach an image from the local images folder to the email 4. Enter recipient details in the To field 5. Enter text in the Subject field 6. Enter text in the Compose field 7. Click the Send mail button

Known working APK version: 7.6.18.160170480

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

recipient : str

The email address of the recipient. Setting a void address will stop any mesage failures clogging up your device inbox

default: 'wa-devnull@mailinator.com'

test_image : str

An image to be copied onto the device that will be attached to the email

default: 'uxperf_1600x1200.jpg'

googlemap

Navigation app.

Stock map provided by Google Inc. Based on revent, we can use this workload to do multiple tasks such as navigation usecases, swipe & pinch etc.

Provided revent is for Odriod XU3 for navigation use case. For running on other devices, we need to build revent.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

googlephotos

A workload to perform standard productivity tasks with Google Photos. The workload carries out various tasks, such as browsing images, performing zooms, and post-processing the image.

Test description:

1. Four images are copied to the device
2. The application is started in offline access mode
3. Gestures are performed to pinch zoom in and out of the selected image
4. The colour of a selected image is edited by selecting the colour menu, incrementing the colour, resetting the colour and decrementing the colour using the seek bar.
5. A crop test is performed on a selected image. UiAutomator does not allow the selection of the crop markers so the image is tilted positively, reset and then tilted negatively to get a similar cropping effect.
6. A rotate test is performed on a selected image, rotating anticlockwise 90 degrees, 180 degrees and 270 degrees.

Known working APK version: 1.21.0.123444480

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

test_images : list_of_strs

A list of four JPEG and/or PNG files to be pushed to the device. Absolute file paths may be used but tilde expansion must be escaped.

constraint: len(unique(value)) == 4

default: ['uxperf_1200x1600.png', 'uxperf_1600x1200.jpg', 'uxperf_2448x3264.png', 'uxperf_3264x2448.jpg']

googleplaybooks

A workload to perform standard productivity tasks with googleplaybooks. This workload performs various tasks, such as searching for a book title online, browsing through a book, adding and removing notes, word searching, and querying information about the book.

Test description: 1. Open Google Play Books application 2. Dismisses sync operation (if applicable) 3. Searches for a book title 4. Adds books to library if not already present 5. Opens ‘My Library’ contents 6. Opens selected book 7. Gestures are performed to swipe between pages and pinch zoom in and out of a page 8. Selects a specified chapter based on page number from the navigation view 9. Selects a word in the centre of screen and adds a test note to the page 10. Removes the test note from the page (clean up) 11. Searches for the number of occurrences of a common word throughout the book 12. Switches page styles from ‘Day’ to ‘Night’ to ‘Sepia’ and back to ‘Day’ 13. Uses the ‘About this book’ facility on the currently selected book

NOTE: This workload requires a network connection (ideally, wifi) to run,

a Google account to be setup on the device, and payment details for the account. Free books require payment details to have been setup otherwise it fails. Tip: Install the ‘Google Opinion Rewards’ app to bypass the need to enter valid card/bank detail.

Known working APK version: 3.13.17

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

search_book_title : str

The book title to search for within Google Play Books archive. The book must either be already in the account’s library, or free to purchase.

default: 'Nikola Tesla: Imagination and the Man That Invented the 20th Century'

library_book_title : str

The book title to search for within My Library. The Library name can differ (usually shorter) to the Store name. If left blank, the search_book_title will be used.

default: 'Nikola Tesla'

select_chapter_page_number : integer

The Page Number to search for within a selected book’s Chapter list. Note: Accepts integers only.

default: 4

search_word : str

The word to search for within a selected book. Note: Accepts single words only.

default: 'the'

account : str

If you are running this workload on a device which has more than one Google account setup, then this parameter is used to select which account to select when prompted. The account requires the book to have already been purchased or payment details already associated with the account. If omitted, the first account in the list will be selected if prompted.

googleslides

A workload to perform standard productivity tasks with Google Slides. The workload carries out various tasks, such as creating a new presentation, adding text, images, and shapes, as well as basic editing and playing a slideshow. This workload should be able to run without a network connection.

There are two main scenarios:

1. create test: a presentation is created in-app and some editing done on it,
2. load test: a pre-existing PowerPoint file is copied onto the device for testing.

— create — Create a new file in the application and perform basic editing on it. This test also requires an image file specified by the param test_image to be copied onto the device.

Test description:

1. Start the app and skip the welcome screen. Dismiss the work offline banner if present.
2. Go to the app settings page and enables PowerPoint compatibility mode. This allows PowerPoint files to be created inside Google Slides.
3. Create a new PowerPoint presentation in the app (PPT compatibility mode) with a title slide and save it to device storage.
4. Insert another slide and to it insert the pushed image by picking it from the gallery.
5. Insert a final slide and add a shape to it. Resize and drag the shape to modify it.
6. Finally, navigate back to the documents list.

— load — Copy a PowerPoint presentation onto the device to test slide navigation. The PowerPoint file to be copied is given by test_file.

Test description:

1. From the documents list (following the create test), open the specified PowerPoint by navigating into device storage and wait for it to be loaded.
2. A navigation test is performed while the file is in editing mode (i.e. not slideshow). swiping forward to the next slide until slide_count swipes are performed.
3. While still in editing mode, the same action is done in the reverse direction back to the first slide.
4. Enter presentation mode by selecting to play the slideshow.
5. Swipe forward to play the slideshow, for a maximum number of slide_count swipes.
6. Finally, repeat the previous step in the reverse direction while still in presentation mode, navigating back to the first slide.

NOTE: There are known issues with the reliability of this workload on some targets. It MAY NOT ALWAYS WORK on your device. If you do run into problems, it might help to set do_text_entry parameter to False.

Known working APK version: 1.7.032.06

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

test_image : str

An image to be copied onto the device that will be embedded in the PowerPoint file as part of the test.

default: 'uxperf_1600x1200.jpg'

test_file : str

If specified, the workload will copy the PowerPoint file to be used for testing onto the device. Otherwise, a file will be created inside the app.

default: 'uxperf_test_doc.pptx'

slide_count : integer

Number of slides in aforementioned local file. Determines number of swipe actions when playing slide show.

default: 5

do_text_entry : boolean

If set to True, will attempt to enter text in the first slide as part of the test. Currently seems to be problematic on some devices, most notably Samsung devices.

default: True

gunbros2

Gun Bros. 2 game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

hackbench

Hackbench runs a series of tests for the Linux scheduler.

For details, go to: https://github.com/linux-test-project/ltp/

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

datasize : integer

Message size in bytes.

default: 100

groups : integer

Number of groups.

default: 10

loops : integer

Number of loops.

default: 100

fds : integer

Number of file descriptors.

default: 40

extra_params : str

Extra parameters to pass in. See the hackbench man page or type hackbench –help for list of options.

duration : integer

Test duration in seconds.

default: 30

homescreen

A workload that goes to the home screen and idles for the the specified duration.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

duration : integer

Specifies the duration, in seconds, of this workload.

default: 20

hwuitest

Tests UI rendering latency on android devices

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

test : caseless_string

The test to run:

• 'shadowgrid': creates a grid of rounded rects that cast shadows, high CPU & GPU load
• 'rectgrid': creates a grid of 1x1 rects
• 'oval': draws 1 oval

allowed values: 'shadowgrid', 'rectgrid', 'oval'

default: 'shadowgrid'

loops : integer

The number of test iterations.

default: 3

frames : integer

The number of frames to run the test over.

default: 150

idle

Do nothing for the specified duration.

On android devices, this may optionally stop the Android run time, if stop_android is set to True.

Note

This workload requires the device to be rooted.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

duration : integer

Specifies the duration, in seconds, of this workload.

default: 20

stop_android : boolean

Specifies whether the Android run time should be stopped. (Can be set only for Android devices).

iozone

Iozone is a filesystem benchmark that runs a series of disk I/O performance tests.

Here is a list of tests that you can run in the iozone workload. The descriptions are from the official iozone document.

0 - Write Test

Measure performance of writing a new file. Other tests rely on the file written by this, so it must always be enabled (WA will automatically neable this if not specified).

1 - Rewrite Test

Measure performance of writing an existing file.

2 - Read Test

Measure performance of reading an existing file.

3 - Reread Test

Measure performance of rereading an existing file.

4 - Random Read Test

Measure performance of reading a file by accessing random locations within the file.

5 - Random Write Test

Measure performance of writing a file by accessing random locations within the file.

6 - Backwards Read Test

Measure performance of reading a file backwards.

7 - Record Rewrite Test

Measure performance of writing and rewriting a particular spot within the file.

8 - Strided Read Test

Measure performance of reading a file with strided access behavior.

9 - Fwrite Test

Measure performance of writing a file using the library function fwrite() that performances buffered write operations.

10 - Frewrite Test

Measure performance of writing a file using the the library function fwrite() that performs buffered and blocked write operations.

11 - Fread Test

Measure performance of reading a file using the library function fread() that performs buffered and blocked read operations.

12 - Freread Test

Same as the Fread Test except the current file being read was read previously sometime in the past.

By default, iozone will run all tests in auto mode. To run specific tests, they must be written in the form of:

[0,1,4,5]

Please enable classifiers in your agenda or config file in order to display the results properly in the results.csv file.

The official website for iozone is at www.iozone.org.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

tests : list_of_ints

List of performance tests to run.

allowed values: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12

auto_mode : boolean

Run tests in auto mode.

default: True

timeout : integer

Timeout for the workload.

default: 14400

file_size : integer

Fixed file size.

record_length : integer

Fixed record length.

threads : integer

Number of threads

other_params : str

Other parameter. Run iozone -h to see list of options.

ironman3

Iron Man 3 game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

krazykart

Krazy Kart Racing game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

linpack

The LINPACK Benchmarks are a measure of a system’s floating point computing power.

http://en.wikipedia.org/wiki/LINPACK_benchmarks

From the article:

Introduced by Jack Dongarra, they measure how fast a computer solves a dense n by n system of linear equations Ax = b, which is a common task in engineering.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

output_file : str

On-device output file path.

linpack-cli

linpack benchmark with a command line interface

Benchmarks FLOPS (floating point operations per second).

This is the oldschool version of the bencmark. Source may be viewed here:

http://www.netlib.org/benchmark/linpackc.new

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

array_size : integer

size of arrays to be used by the benchmark.

default: 200

lmbench

Run a subtest from lmbench, a suite of portable ANSI/C microbenchmarks for UNIX/POSIX.

In general, lmbench measures two key features: latency and bandwidth. This workload supports a subset of lmbench tests. lat_mem_rd can be used to measure latencies to memory (including caches). bw_mem can be used to measure bandwidth to/from memory over a range of operations.

Further details, and source code are available from:

http://sourceforge.net/projects/lmbench/.

See lmbench/bin/README for license details.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

test : str

Specifies an lmbench test to run.

allowed values: 'lat_mem_rd', 'bw_mem'

default: 'lat_mem_rd'

stride : list_or_type

Stride for lat_mem_rd test. Workload will iterate over one or more integer values.

default: [128]

thrash : boolean

Sets -t flag for lat_mem_rd_test

default: True

size : list_or_string

Data set size for lat_mem_rd bw_mem tests.

default: '4m'

mem_category : list_or_string

List of memory catetories for bw_mem test.

default: ('rd', 'wr', 'cp', 'frd', 'fwr', 'fcp', 'bzero', 'bcopy')

parallelism : integer

Parallelism flag for tests that accept it.

warmup : integer

Warmup flag for tests that accept it.

repetitions : integer

Repetitions flag for tests that accept it.

force_abi : str

Override device abi with this value. Can be used to force arm32 on 64-bit devices.

run_timeout : integer

Timeout for execution of the test.

default: 900

times : integer

Specifies the number of times the benchmark will be run in a “tight loop”, i.e. without performaing setup/teardown inbetween. This parameter is distinct from “repetitions”, as the latter takes place within the benchmark and produces a single result.

constraint: value > 0

default: 1

taskset_mask : integer

Specifies the CPU mask the benchmark process will be pinned to.

manual

Yields control to the user, either for a fixed period or based on user input, to perform custom operations on the device, about which workload automation does not know of.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

duration : integer

Control of the devices is yielded for the duration (in seconds) specified. If not specified, user_triggered is assumed.

user_triggered : boolean

If True, WA will wait for user input after starting the workload; otherwise fixed duration is expected. Defaults to True if duration is not specified, and False otherwise.

view : str

Specifies the View of the workload. This enables instruments that require a View to be specified, such as the fps instrument. This is required for using “SurfaceFlinger” to collect FPS statistics and is primarily used on devices pre API level 23

default: 'SurfaceView'

package : str

Specifies the package name of the workload. This enables instruments that require a Package to be specified, such as the fps instrument. This allows for “gfxinfo” to be used and is the preferred method of collection for FPS statistics on devices API level 23+

enable_logcat : boolean

If True, manual workload will collect logcat as part of the results.

memcpy

Runs memcpy in a loop.

This will run memcpy in a loop for a specified number of times on a buffer of a specified size. Additionally, the affinity of the test can be set to one or more specific cores.

This workload is single-threaded. It genrates no scores or metrics by itself.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

buffer_size : integer

Specifies the size, in bytes, of the buffer to be copied.

default: 5242880

iterations : integer

Specfies the number of iterations that will be performed.

default: 1000

cpus : list

A list of integers specifying ordinals of cores to which the affinity of the test process should be set. If not specified, all avaiable cores will be used.

nenamark

NenaMark is an OpenGL-ES 2.0 graphics performance benchmark for Android devices.

http://nena.se/nenamark_story

From the website:

The NenaMark2 benchmark scene averages about 45k triangles, with a span between 26k and 68k triangles. It averages 96 batches per frame and contains about 15 Mb of texture data (non-packed).

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

duration : integer

Number of seconds to wait before considering the benchmark finished

default: 120

octaned8

Runs the Octane d8 benchmark.

This workload runs d8 binaries built from source and placed in the dependencies folder along with test assets from https://github.com/chromium/octane which also need to be placed in an assets folder within the dependencies folder.

Original source from:

https://github.com/v8/v8/wiki/D8%20on%20Android

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

run_timeout : integer

Timeout, in seconds, for the test execution.

default: 180

peacekeeper

Peacekeeper is a free and fast browser test that measures a browser’s speed.

Note

This workload requires a network connection as well as support for one of the two currently-supported browsers. Moreover, TC2 has compatibility issue with chrome

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

browser : str

The browser to be benchmarked.

allowed values: 'firefox', 'chrome'

default: 'firefox'

output_file : str

The result URL of peacekeeper benchmark will be written into this file on device after completion of peacekeeper benchmark. Defaults to peacekeeper.txt in the device’s working_directory.

peacekeeper_url : str

The URL to run the peacekeeper benchmark.

default: 'http://peacekeeper.futuremark.com/run.action'

power_loadtest

power_LoadTest (part of ChromeOS autotest suite) continuously cycles through a set of browser-based activities and monitors battery drain on a device.

Note

This workload must be run inside a CromeOS SDK chroot.

See: https://www.chromium.org/chromium-os/testing/power-testing

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

board : str

The name of the board to be used for the test. If this is not specified, BOARD environment variable will be used.

variant : str

The variant of the test to run; If not specified, the full power_LoadTest will run (until the device battery is drained). The only other variant available in the vanilla test is “1hour”, but further variants may be added by providing custom control files.

test_that_args : arguments

Extra arguments to be passed to test_that_invocation.

run_timeout : integer

Timeout, in seconds, for the test execution.

default: 86400

quadrant

Quadrant is a benchmark for mobile devices, capable of measuring CPU, memory, I/O and 3D graphics performance.

http://www.aurorasoftworks.com/products/quadrant

From the website: Quadrant outputs a score for the following categories: 2D, 3D, Mem, I/O, CPU , Total.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

real-linpack

This version of Linpack <http://en.wikipedia.org/wiki/LINPACK_benchmarks> was developed by Dave Butcher. RealLinpack tries to find the number of threads that give you the maximum linpack score.

RealLinpack runs 20 runs of linpack for each number of threads and calculates the mean and confidence. It stops when the score’s confidence interval drops below the current best score interval. That is, when (current_score + confidence) < (best_score - best_score_confidence)

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

max_threads : integer

The maximum number of threads that real linpack will try.

constraint: value > 0

default: 16

realracing3

Real Racing 3 game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

recentfling

Tests UI jank on android devices.

For this workload to work, recentfling.sh and defs.sh must be placed in ~/.workload_automation/dependencies/recentfling/. These can be found in the AOSP Git repository.

To change the apps that are opened at the start of the workload you will need to modify the defs.sh file. You will need to add your app to dfltAppList and then add a variable called {app_name}Activity with the name of the activity to launch (where {add_name} is the name you put into dfltAppList).

You can get a list of activities available on your device by running adb shell pm list packages -f

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

loops : integer

The number of test iterations.

default: 3

start_apps : boolean

If set to False,no apps will be started before flinging through the recent apps list (in which the assumption is there are already recently started apps in the list.

default: True

device_name : str

If set, recentfling will use the fling parameters for this device instead of automatically guessing the device. This can also be used if the device is not supported by recentfling, but its screensize is similar to that of one that is supported.

For possible values, check your recentfling.sh. At the time of writing, valid values are: ‘shamu’, ‘hammerhead’, ‘angler’, ‘ariel’, ‘mtp8996’, ‘bullhead’ or ‘volantis’.

rt-app

A test application that simulates configurable real-time periodic load.

rt-app is a test application that starts multiple periodic threads in order to simulate a real-time periodic load. It supports SCHED_OTHER, SCHED_FIFO, SCHED_RR as well as the AQuoSA framework and SCHED_DEADLINE.

The load is described using JSON-like config files. Below are a couple of simple examples.

Simple use case which creates a thread that run 1ms then sleep 9ms until the use case is stopped with Ctrl+C:

{
    "tasks" : {
        "thread0" : {
            "loop" : -1,
            "run" :   20000,
            "sleep" : 80000
        }
    },
    "global" : {
        "duration" : 2,
        "calibration" : "CPU0",
        "default_policy" : "SCHED_OTHER",
        "pi_enabled" : false,
        "lock_pages" : false,
        "logdir" : "./",
        "log_basename" : "rt-app1",
        "ftrace" : false,
        "gnuplot" : true,
    }
}

Simple use case with 2 threads that runs for 10 ms and wake up each other until the use case is stopped with Ctrl+C

{
    "tasks" : {
        "thread0" : {
            "loop" : -1,
            "run" :     10000,
            "resume" : "thread1",
            "suspend" : "thread0"
        },
        "thread1" : {
            "loop" : -1,
            "run" :     10000,
            "resume" : "thread0",
            "suspend" : "thread1"
        }
    }
}

Please refer to the existing configs in $WA_ROOT/wlauto/workloads/rt_app/use_case for more examples.

The version of rt-app currently used with this workload contains enhancements and modifications done by Linaro. The source code for this version may be obtained here:

http://git.linaro.org/power/rt-app.git

The upstream version of rt-app is hosted here:

https://github.com/scheduler-tools/rt-app

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

config : str

Use case configuration file to run with rt-app. This may be either the name of one of the “standard” configurations included with the workload. or a path to a custom JSON file provided by the user. Either way, the “.json” extension is implied and will be added automatically if not specified in the argument.

The following is the list of standard configurations currently included with the workload: browser-long.json, spreading-tasks.json, mp3-short.json, camera-short.json, browser-short.json, mp3-long.json, video-long.json, camera-long.json, taskset.json, video-short.json

default: 'taskset'

duration : integer

Duration of the workload execution in Seconds. If specified, this will override the corresponding parameter in the JSON config.

taskset_mask : integer

Constrain execution to specific CPUs.

uninstall_on_exit : boolean

If set to True, rt-app binary will be uninstalled from the device at the end of the run.

force_install : boolean

If set to True, rt-app binary will always be deployed to the target device at the beginning of the run, regardless of whether it was already installed there.

shellscript

Runs an arbitrary shellscript on the device.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

script_file : str (mandatory)

The path (on the host) to the shell script file. This must be an absolute path (though it may contain ~).

argstring : str

A string that should contain arguments passed to the script.

timeout : integer

Timeout, in seconds, for the script run time.

default: 60

skype

A workload to perform standard productivity tasks within Skype. The workload logs in to the Skype application, selects a recipient from the contacts list and then initiates either a voice or video call.

Test description:

1. Open Skype application
2. Log in to a pre-defined account
3. Select a recipient from the Contacts list
4. Initiate either a voice or video call for duration time (in seconds) Note: The actual duration of the call may not match exactly the intended duration due to the uiautomation overhead.

Skype Setup

• You must have a Skype account set up and its credentials passed as parameters into this workload
• The contact to be called must be added (and has accepted) to the account. It’s possible to have multiple contacts in the list, however the contact to be called must be visible on initial navigation to the list.
• For video calls the contact must be able to received the call. This means that there must be a Skype client running (somewhere) with the contact logged in and that client must have been configured to auto-accept calls from the account on the device (how to set this varies between different versions of Skype and between platforms – please search online for specific instructions). https://support.skype.com/en/faq/FA3751/can-i-automatically-answer-all-my-calls-with-video-in-skype-for-windows-desktop

Known working APK version: 7.01.0.669

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

login_name : str (mandatory)

Account to use when logging into the device from which the call will be made

login_pass : str (mandatory)

Password associated with the account to log into the device

contact_name : str

This is the contact display name as it appears in the people list

default: 'Echo / Sound Test Service'

duration : integer

This is the target duration of the call in seconds

default: 10

action : str

Action to take - either voice call (default) or video

allowed values: 'voice', 'video'

default: 'voice'

smartbench

Smartbench is a multi-core friendly benchmark application that measures the overall performance of an android device. It reports both Productivity and Gaming Index.

https://play.google.com/store/apps/details?id=com.smartbench.twelve&hl=en

From the website:

It will be better prepared for the quad-core world. Unfortunately this also means it will run slower on older devices. It will also run slower on high-resolution tablet devices. All 3D tests are now rendered in full native resolutions so naturally it will stress hardware harder on these devices. This also applies to higher resolution hand-held devices.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

spec2000

SPEC2000 benchmarks measuring processor, memory and compiler.

http://www.spec.org/cpu2000/

From the web site:

SPEC CPU2000 is the next-generation industry-standardized CPU-intensive benchmark suite. SPEC designed CPU2000 to provide a comparative measure of compute intensive performance across the widest practical range of hardware. The implementation resulted in source code benchmarks developed from real user applications. These benchmarks measure the performance of the processor, memory and compiler on the tested system.

Note

At the moment, this workload relies on pre-built SPEC binaries (included in an asset bundle). These binaries must be built according to rules outlined here:

http://www.spec.org/cpu2000/docs/runrules.html#toc_2.0

in order for the results to be valid SPEC2000 results.

Note

This workload does not attempt to generate results in an admissible SPEC format. No metadata is provided (though some, but not all, of the required metdata is collected by WA elsewhere). It is upto the user to post-process results to generated SPEC-admissible results file, if that is their intention.

base vs peak

SPEC2000 defines two build/test configuration: base and peak. Base is supposed to use basic configuration (e.g. default compiler flags) with no tuning, and peak is specifically optimized for a system. Since this workload uses externally-built binaries, there is no way for WA to be sure what configuration is used – the user is expected to keep track of that. Be aware that base/peak also come with specific requirements for the way workloads are run (e.g. how many instances on multi-core systems):

http://www.spec.org/cpu2000/docs/runrules.html#toc_3

These are not enforced by WA, so it is again up to the user to ensure that correct workload parameters are specfied inthe agenda, if they intend to collect “official” SPEC results. (Those interested in collecting official SPEC results should also note that setting runtime parameters would violate SPEC runs rules that state that no configuration must be done to the platform after boot).

bundle structure

This workload expects the actual benchmark binaries to be provided in a tarball “bundle” that has a very specific structure. At the top level of the tarball, there should be two directories: “fp” and “int” – for each of the SPEC2000 categories. Under those, there is a sub-directory per benchmark. Each benchmark sub-directory contains three sub-sub-directorie:

• “cpus” contains a subdirectory for each supported cpu (e.g. a15) with a single executable binary for that cpu, in addition to a “generic” subdirectory that has not been optimized for a specific cpu and should run on any ARM system.
• “data” contains all additional files (input, configuration, etc) that the benchmark executable relies on.
• “scripts” contains one or more one-liner shell scripts that invoke the benchmark binary with appropriate command line parameters. The name of the script must be in the format <benchmark name>[.<variant name>].sh, i.e. name of benchmark, optionally followed by variant name, followed by “.sh” extension. If there is more than one script, then all of them must have a variant; if there is only one script the it should not contain a variant.

A typical bundle may look like this:

|- fp
|  |-- ammp
|  |   |-- cpus
|  |   |   |-- generic
|  |   |   |   |-- ammp
|  |   |   |-- a15
|  |   |   |   |-- ammp
|  |   |   |-- a7
|  |   |   |   |-- ammp
|  |   |-- data
|  |   |   |-- ammp.in
|  |   |-- scripts
|  |   |   |-- ammp.sh
|  |-- applu
.  .   .
.  .   .
.  .   .
|- int
.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

benchmarks : list_or_string

Specifies the SPEC benchmarks to run.

mode : str

SPEC benchmarks can report either speed to execute or throughput/rate. In the latter case, several “threads” will be spawned.

allowed values: 'speed', 'rate'

default: 'speed'

number_of_threads : integer

Specify the number of “threads” to be used in ‘rate’ mode. (Note: on big.LITTLE systems this is the number of threads, for each cluster).

force_extract_assets : boolean

if set to True, will extract assets from the bundle, even if they are already extracted. Note: this option implies force_push_assets.

force_push_assets : boolean

If set to True, assets will be pushed to device even if they’re already present.

timeout : integer

Timeout, in seconds, for the execution of single spec test.

default: 1200

sqlitebm

Measures the performance of the sqlite database. It determines within what time the target device processes a number of SQL queries.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

stream

Measures memory bandwidth.

The original source code be found on: https://www.cs.virginia.edu/stream/FTP/Code/

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

threads : integer

The number of threads to execute if OpenMP is enabled

stress_ng

stress-ng will stress test a computer system in various selectable ways. It was designed to exercise various physical subsystems of a computer as well as the various operating system kernel interfaces.

stress-ng can also measure test throughput rates; this can be useful to observe performance changes across different operating system releases or types of hardware. However, it has never been intended to be used as a precise benchmark test suite, so do NOT use it in this manner.

The official website for stress-ng is at:

http://kernel.ubuntu.com/~cking/stress-ng/

Source code are available from:

http://kernel.ubuntu.com/git/cking/stress-ng.git/

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

stressor : str

Stress test case name. The cases listed in allowed values come from the stable release version 0.01.32. The binary included here compiled from dev version 0.06.01. Refer to man page for the definition of each stressor.

allowed values: 'cpu', 'io', 'fork', 'switch', 'vm', 'pipe', 'yield', 'hdd', 'cache', 'sock', 'fallocate', 'flock', 'affinity', 'timer', 'dentry', 'urandom', 'sem', 'open', 'sigq', 'poll'

default: 'cpu'

threads : integer

The number of workers to run. Specifying a negative or zero value will select the number of online processors.

duration : integer

Timeout for test execution in seconds

default: 60

sysbench

SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load.

The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.

Features of SysBench

• file I/O performance
• scheduler performance
• memory allocation and transfer speed
• POSIX threads implementation performance
• database server performance

See: https://github.com/akopytov/sysbench

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

timeout : integer

timeout for workload execution (adjust from default if running on a slow device and/or specifying a large value for max_requests

default: 300

test : str

sysbench test to run

allowed values: 'fileio', 'cpu', 'memory', 'threads', 'mutex'

default: 'cpu'

threads : integer

The number of threads sysbench will launch

default: 8

num_threads : integer

The number of threads sysbench will launch, overrides threads (old parameter name)

max_requests : integer

The limit for the total number of requests.

max_time : integer

The limit for the total execution time. If neither this nor max_requests is specified, this will default to 30 seconds.

file_test_mode : str

File test mode to use. This should only be specified if test is "fileio"; if that is the case and file_test_mode is not specified, it will default to "seqwr" (please see sysbench documentation for explanation of various modes).

allowed values: 'seqwr', 'seqrewr', 'seqrd', 'rndrd', 'rndwr', 'rndrw'

cmd_params : str

Additional parameters to be passed to sysbench as a single stiring

force_install : boolean

Always install binary found on the host, even if already installed on device

default: True

taskset_mask : integer

The processes spawned by sysbench will be pinned to cores as specified by this parameter

telemetry

Executes Google’s Telemetery benchmarking framework

Url: https://www.chromium.org/developers/telemetry

From the web site:

Telemetry is Chrome’s performance testing framework. It allows you to perform arbitrary actions on a set of web pages and report metrics about it. The framework abstracts:

• Launching a browser with arbitrary flags on any platform.
• Opening a tab and navigating to the page under test.
• Fetching data via the Inspector timeline and traces.
• Using Web Page Replay to cache real-world websites so they don’t change when used in benchmarks.

Design Principles

• Write one performance test that runs on all platforms - Windows, Mac, Linux, Chrome OS, and Android for both Chrome and ContentShell.
• Runs on browser binaries, without a full Chromium checkout, and without having to build the browser yourself.
• Use WebPageReplay to get repeatable test results.
• Clean architecture for writing benchmarks that keeps measurements and use cases separate.
• Run on non-Chrome browsers for comparative studies.

This instrument runs telemetry via its run_benchmark script (which must be in PATH or specified using run_benchmark_path parameter) and parses metrics from the resulting output.

device setup

The device setup will depend on whether you’re running a test image (in which case little or no setup should be necessary)

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

run_benchmark_path : str

This is the path to run_benchmark script which runs a Telemetry benchmark. If not specified, WA will look for Telemetry in its dependencies; if not found there, Telemetry will be downloaded.

test : str

Specifies the telemetry test to run.

default: 'page_cycler.top_10_mobile'

run_benchmark_params : str

Additional paramters to be passed to run_benchmark.

run_timeout : integer

Timeout for execution of the test.

default: 900

extract_fps : boolean

if True, FPS for the run will be computed from the trace (must be enabled).

target_config : str

Manually specify target configuration for telemetry. This must contain –browser option plus any addition options Telemetry requires for a particular target (e.g. –device or –remote)

templerun

Templerun game.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

thechase

The Chase demo showcasing the capabilities of Unity game engine.

This demo, is a static video-like game demo, that demonstrates advanced features of the unity game engine. It loops continuously until terminated.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

duration : integer

Duration, in seconds, note that the demo loops the same (roughly) 60 second sceene until stopped.

default: 70

truckerparking3d

Trucker Parking 3D game.

(yes, apparently that’s a thing…)

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 500

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

idle_time : integer

The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file).

check_states : boolean

Use visual state detection to verify the state of the workload after setup and run

assets_push_timeout : integer

Timeout used during deployment of the assets package (if there is one).

default: 500

vellamo

Android benchmark designed by Qualcomm.

Vellamo began as a mobile web benchmarking tool that today has expanded to include three primary chapters. The Browser Chapter evaluates mobile web browser performance, the Multicore chapter measures the synergy of multiple CPU cores, and the Metal Chapter measures the CPU subsystem performance of mobile processors. Through click-and-go test suites, organized by chapter, Vellamo is designed to evaluate: UX, 3D graphics, and memory read/write and peak bandwidth performance, and much more!

Note: Vellamo v3.0 fails to run on Juno

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

version : str

Specify the version of Vellamo to be run. If not specified, the latest available version will be used.

allowed values: '3.2.4', '2.0.3', '3.0'

default: '3.2.4'

benchmarks : list_of_strs

Specify which benchmark sections of Vellamo to be run. Only valid on version 3.0 and newer. NOTE: Browser benchmark can be problematic and seem to hang,just wait and it will progress after ~5 minutes

allowed values: 'Browser', 'Metal', 'Multi'

default: ['Browser', 'Metal', 'Multi']

browser : integer

Specify which of the installed browsers will be used for the tests. The number refers to the order in which browsers are listed by Vellamo. E.g. 1 will select the first browser listed, 2 – the second, etc. Only valid for version 3.0.

default: 1

video

Plays a video file using the standard android video player for a predetermined duration.

The video can be specified either using resolution workload parameter, in which case Big Buck Bunny MP4 video of that resolution will be downloaded and used, or using filename parameter, in which case the video file specified will be used.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

play_duration : integer

Playback duration of the video file. This become the duration of the workload.

default: 20

resolution : str

Specifies which resolution video file to play.

allowed values: '480p', '720p', '1080p'

default: '720p'

filename : str

The name of the video file to play. This can be either a path to the file anywhere on your file system, or it could be just a name, in which case, the workload will look for it in ~/.workloads_automation/dependency/video Note: either resolution or filename should be specified, but not both!

force_dependency_push : boolean

If true, video will always be pushed to device, regardless of whether the file is already on the device. Default is False.

videostreaming

Uses the FREEdi video player to search, stream and play the specified video content from YouTube.

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

video_name : str

Name of the video to be played.

resolution : str

Resolution of the video to be played. If video_name is setthis setting will be ignored

allowed values: '320p', '720p', '1080p'

default: '320p'

sampling_interval : integer

Time interval, in seconds, after which the status of the video playback to be monitoreThe elapsed time of the video playback is monitored after after every sampling_interval seconds and compared against the actual time elapsed and the previous sampling point. If the video elapsed time is less that (sampling time - tolerance) , then the playback is aborted as the video has not been playing continuously.

default: 20

tolerance : integer

Specifies the amount, in seconds, by which sampling time is allowed to deviate from elapsed video playback time. If the delta is greater than this value (which could happen due to poor network connection), workload result will be invalidated.

default: 3

run_timeout : integer

The duration in second for which to play the video

default: 200

youtube

A workload to perform standard productivity tasks within YouTube.

The workload plays a video from the app, determined by the video_source parameter. While the video is playing, a some common actions are done such as video seeking, pausing playback and navigating the comments section.

Test description: The video_source parameter determines where the video to be played will be found in the app. Possible values are search, home, my_videos, and trending.

-A. search - Goes to the search view, does a search for the given term, and plays the

first video in the results. The parameter search_term must also be provided in the agenda for this to work. This is the default mode.

-B. home - Scrolls down once on the app’s home page to avoid ads (if present, would be

first video), then select and plays the video that appears at the top of the list.

-C. my_videos - Goes to the ‘My Videos’ section of the user’s account page and plays a

video from there. The user must have at least one uploaded video for this to work.

-D. trending - Goes to the ‘Trending Videos’ section of the app, and plays the first

video in the trending videos list.

For the selected video source, the following test steps are performed:

1. Navigate to the general app settings page to disable autoplay. This improves test stability and predictability by preventing screen transition to load a new video while in the middle of the test.
2. Select the video from the source specified above, and dismiss any potential embedded advert that may pop-up before the actual video.
3. Let the video play for a few seconds, pause it, then resume.
4. Expand the info card that shows video metadata, then collapse it again.
5. Scroll down to the end of related videos and comments under the info card, and then back up to the start. A maximum of 5 swipe actions is performed in either direction.

Known working APK version: 11.19.56

parameters

modules : list

Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension.

install_timeout : integer

Timeout for the installation of the apk.

default: 300

check_apk : boolean

When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered.

default: True

force_install : boolean

Always re-install the APK, even if matching version is found already installed on the device. Runs adb install -r to ensure existing APK is replaced. When this is set, check_apk is ignored.

uninstall_apk : boolean

If True, will uninstall workload’s APK as part of teardown.

exact_abi : boolean

If True, workload will check that the APK matches the target device ABI, otherwise any APK found will be used.

clear_data_on_reset : boolean

If set to False, this will prevent WA from clearing package data for this workload prior to running it.

default: True

markers_enabled : boolean

If True, UX_PERF action markers will be emitted to logcat during the test run.

clean_assets : boolean

If True pushed assets will be deleted at the end of each iteration

force_push_assets : boolean

If True always push assets on each iteration, even if the assets already exists in the device path

video_source : str

Determines where to play the video from. This can either be from the YouTube home, my videos section, trending videos or found in search.

allowed values: 'home', 'my_videos', 'search', 'trending'

default: 'search'

search_term : str

The search term to use when video_source is set to search. Ignored otherwise.

default: 'Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film'