.. _workloads: Workloads ========= adobereader ----------- The Adobe Reader workflow carries out the following typical productivity tasks. Test description: 1. Open a local file on the device 2. Gestures test: 2.1. Swipe down across the central 50% of the screen in 200 x 5ms steps 2.2. Swipe up across the central 50% of the screen in 200 x 5ms steps 2.3. Swipe right from the edge of the screen in 50 x 5ms steps 2.4. Swipe left from the edge of the screen in 50 x 5ms steps 2.5. Pinch out 50% in 100 x 5ms steps 2.6. Pinch In 50% in 100 x 5ms steps 3. Search test: Search ``document_name`` for each string in the ``search_string_list`` 4. Close the document Known working APK version: 16.1 parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path document_name : str The document name to use for the Gesture and Search test. default: ``'uxperf_test_doc.pdf'`` search_string_list : list_of_strs For each string in the list, a document search is performed using the string as the search term. At least one must be provided. constraint: ``len(value) > 0`` default: ``['The quick brown fox jumps over the lazy dog', 'TEST_SEARCH_STRING']`` andebench --------- AndEBench is an industry standard Android benchmark provided by The Embedded Microprocessor Benchmark Consortium (EEMBC). http://www.eembc.org/andebench/about.php From the website: - Initial focus on CPU and Dalvik interpreter performance - Internal algorithms concentrate on integer operations - Compares the difference between native and Java performance - Implements flexible multicore performance analysis - Results displayed in Iterations per second - Detailed log file for comprehensive engineering analysis parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` number_of_threads : integer Number of threads that will be spawned by AndEBench. single_threaded : boolean If ``true``, AndEBench will run with a single thread. Note: this must not be specified if ``number_of_threads`` has been specified. native_only : boolean If ``true``, AndEBench will execute only the native portion of the benchmark. androbench ---------- Measures the storage performance of an Android device. Website: http://www.androbench.org/wiki/AndroBench parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` angrybirds ---------- Angry Birds game. A very popular Android 2D game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` angrybirds_rio -------------- Angry Birds Rio game. The sequel to the very popular Android 2D game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` anomaly2 -------- Anomaly 2 game demo and benchmark. Plays three scenes from the game, benchmarking each one. Scores reported are intended to represent overall perceived quality of the game, based not only on raw FPS but also factors like smoothness. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` antutu ------ AnTuTu Benchmark is an benchmarking tool for Android Mobile Phone/Pad. It can run a full test of a key project, through the "Memory Performance","CPU Integer Performance","CPU Floating point Performance","2D 3D Graphics Performance","SD card reading/writing speed","Database IO" performance testing, and gives accurate analysis for Andriod smart phones. http://www.antutulabs.com/AnTuTu-Benchmark From the website: AnTuTu Benchmark can support the latest quad-core cpu. In reaching the overall and individual scores of the hardware, AnTuTu Benchmark could judge your phone by the scores of the performance of the hardware. By uploading the scores, Benchmark can view your device in the world rankings, allowing points to let you know the level of hardware performance equipment. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` version : str Specify the version of AnTuTu to be run. If not specified, the latest available version will be used. allowed values: ``'3.3.2'``, ``'4.0.3'``, ``'5.3.0'``, ``'6.0.1'`` default: ``'6.0.1'`` times : integer The number of times the benchmark will be executed in a row (i.e. without going through the full setup/teardown process). Note: this does not work with versions prior to 4.0.3. default: ``1`` enable_sd_tests : boolean If ``True`` enables SD card tests in pre version 4 AnTuTu. These tests were know to cause problems on platforms without an SD card. This parameter will be ignored on AnTuTu version 4 and higher. apklaunch --------- Installs and runs a .apk file, waits wait_time_seconds, and tests if the app has started successfully. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. apk_file : str (mandatory) Name to the .apk to run uninstall_required : boolean Set to true if the package should be uninstalled wait_time_seconds : integer Seconds to wait before testing if the app is still alive applaunch --------- This workload launches and measures the launch time of applications for supporting workloads. Currently supported workloads are the ones that implement ``ApplaunchInterface``. For any workload to support this workload, it should implement the ``ApplaunchInterface``. The corresponding java file of the workload associated with the application being measured is executed during the run. The application that needs to be measured is passed as a parametre ``workload_name``. The parameters required for that workload have to be passed as a dictionary which is captured by the parametre ``workload_params``. This information can be obtained by inspecting the workload details of the specific workload. The workload allows to run multiple iterations of an application launch in two modes: 1. Launch from background 2. Launch from long-idle These modes are captured as a parameter applaunch_type. ``launch_from_background`` Launches an application after the application is sent to background by pressing Home button. ``launch_from_long-idle`` Launches an application after killing an application process and clearing all the caches. **Test Description:** - During the initialization and setup, the application being launched is launched for the first time. The jar file of the workload of the application is moved to device at the location ``workdir`` which further implements the methods needed to measure the application launch time. - Run phase calls the UiAutomator of the applaunch which runs in two subphases. A. Applaunch Setup Run: During this phase, welcome screens and dialogues during the first launch of the instrumented application are cleared. B. Applaunch Metric Run: During this phase, the application is launched multiple times determined by the iteration number specified by the parametre ``applaunch_iterations``. Each of these iterations are instrumented to capture the launch time taken and the values are recorded as UXPERF marker values in logfile. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path workload_name : str Name of the uxperf workload to launch default: ``'gmail'`` workload_params : dict parameters of the uxperf workload whose application launch time is measured applaunch_type : str Choose launch_from_long-idle for measuring launch time from long-idle. These two types are described in the class description. allowed values: ``'launch_from_background'``, ``'launch_from_long-idle'`` default: ``'launch_from_background'`` applaunch_iterations : integer Number of iterations of the application launch default: ``1`` report_results : boolean Choose to report results of the application launch time. default: ``True`` appshare -------- Workload to test how responsive a device is when context switching between application tasks. It combines workflows from googlephotos, gmail and skype. ** Setup ** Credentials for the user account used to log into the Skype app have to be provided in the agenda, as well as the display name of the contact to call. For reliable testing, this workload requires a good and stable internet connection, preferably on Wi-Fi. Although this workload attempts to be network independent it requires a network connection (ideally, wifi) to run. This is because the welcome screen UI is dependent on an existing connection. Test description: 1. GooglePhotos is started in offline access mode 1.1. The welcome screen is dismissed 1.2. Any promotion popup is dismissed 1.3. The provided ``test_image`` is selected and displayed 2. The image is then shared across apps to Gmail 2.1. The first run dialogue is dismissed 2.2. Enter recipient details in the To field 2.3. Enter text in the Subject field 2.4. Enter text in the Body field 2.5. Click the Send mail button 3. Return to Googlephotos and login to Skype via share action 4. Return to Googlephotos and share the ``test_image`` with Skype 4.1. Search for the ``skype_contact_name`` from the Contacts list 4.2. Dismiss any update popup that appears 4.3. The image is posted in the Chat parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path test_image : str An image to be copied onto the device that will be shared across multiple apps default: ``'uxperf_1600x1200.jpg'`` email_recipient : str The email address of the recipient to recieve the shared image default: ``'wa-devnull@mailinator.com'`` skype_login_name : str (mandatory) Account to use when logging into skype from which to share the image skype_login_pass : str (mandatory) Password associated with the skype account skype_contact_name : str This is the contact display name as it appears in the people list default: ``'Echo / Sound Test Service'`` audio ----- Audio workload plays an MP3 file using the built-in music player. By default, it plays Canon_in_D_Pieano.mp3 for 30 seconds. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. duration : integer The duration the music will play for in seconds. default: ``30`` audio_file : str The (on-host) path to the audio file to be played. .. note:: If the default file is not present locally, it will be downloaded. default: ``'~/.workload_automation/dependencies/Canon_in_D_Piano.mp3'`` perform_cleanup : boolean If ``True``, workload files on the device will be deleted after execution. clear_file_cache : boolean Clear the the file cache on the target device prior to running the workload. default: ``True`` autotest -------- Executes tests from ChromeOS autotest suite .. note:: This workload *must* be run inside a CromeOS SDK chroot. See: https://www.chromium.org/chromium-os/testing/power-testing parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. test : str (mandatory) The test to be run test_that_args : arguments Extra arguments to be passed to test_that_invocation. run_timeout : integer Timeout, in seconds, for the test execution. default: ``1800`` bbench ------ BBench workload opens the built-in browser and navigates to, and scrolls through, some preloaded web pages and ends the workload by trying to connect to a local server it runs after it starts. It can also play the workload while it plays an audio file in the background. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. with_audio : boolean Specifies whether an MP3 should be played in the background during workload execution. server_timeout : integer Specifies the timeout (in seconds) before the server is stopped. default: ``300`` force_dependency_push : boolean Specifies whether to push dependency files to the device to the device if they are already on it. audio_file : str The (on-host) path to the audio file to be played. This is only used if ``with_audio`` is ``True``. default: ``'~/.workload_automation/dependencies/Canon_in_D_Piano.mp3'`` perform_cleanup : boolean If ``True``, workload files on the device will be deleted after execution. clear_file_cache : boolean Clear the the file cache on the target device prior to running the workload. default: ``True`` browser_package : str Specifies the package name of the device's browser app. default: ``'com.android.browser'`` browser_activity : str Specifies the startup activity name of the device's browser app. default: ``'.BrowserActivity'`` benchmarkpi ----------- Measures the time the target device takes to run and complete the Pi calculation algorithm. http://androidbenchmark.com/howitworks.php from the website: The whole idea behind this application is to use the same Pi calculation algorithm on every Android Device and check how fast that proccess is. Better calculation times, conclude to faster Android devices. This way you can also check how lightweight your custom made Android build is. Or not. As Pi is an irrational number, Benchmark Pi does not calculate the actual Pi number, but an approximation near the first digits of Pi over the same calculation circles the algorithms needs. So, the number you are getting in miliseconds is the time your mobile device takes to run and complete the Pi calculation algorithm resulting in a approximation of the first Pi digits. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` blogbench --------- Blogbench is a portable filesystem benchmark that tries to reproduce the load of a real-world busy file server. Blogbench stresses the filesystem with multiple threads performing random reads, writes and rewrites in order to get a realistic idea of the scalability and the concurrency a system can handle. Source code are available from: https://download.pureftpd.org/pub/blogbench/ parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. iterations : integer The number of iterations to run default: ``30`` caffeinemark ------------ CaffeineMark is a series of tests that measure the speed of Java programs running in various hardware and software configurations. http://www.benchmarkhq.ru/cm30/info.html From the website: CaffeineMark scores roughly correlate with the number of Java instructions executed per second, and do not depend significantly on the the amount of memory in the system or on the speed of a computers disk drives or internet connection. The following is a brief description of what each test does: - Sieve: The classic sieve of eratosthenes finds prime numbers. - Loop: The loop test uses sorting and sequence generation as to measure compiler optimization of loops. - Logic: Tests the speed with which the virtual machine executes decision-making instructions. - Method: The Method test executes recursive function calls to see how well the VM handles method calls. - Float: Simulates a 3D rotation of objects around a point. - Graphics: Draws random rectangles and lines. - Image: Draws a sequence of three graphics repeatedly. - Dialog: Writes a set of values into labels and editboxes on a form. The overall CaffeineMark score is the geometric mean of the individual scores, i.e., it is the 9th root of the product of all the scores. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` cameracapture ------------- Uses in-built Android camera app to take photos. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. no_of_captures : integer Number of photos to be taken. default: ``5`` time_between_captures : integer Time, in seconds, between two consecutive camera clicks. default: ``5`` camerarecord ------------ Uses in-built Android camera app to record the video for given interval of time. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. recording_time : integer The video recording time in seconds. default: ``60`` recording_mode : str The video recording mode. allowed values: ``'normal'``, ``'slow_motion'`` default: ``'normal'`` castlebuilder ------------- Castle Builder game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` castlemaster ------------ Castle Master v1.09 game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` cfbench ------- CF-Bench is (mainly) CPU and memory benchmark tool specifically designed to be able to handle multi-core devices, produce a fairly stable score, and test both native as well managed code performance. https://play.google.com/store/apps/details?id=eu.chainfire.cfbench&hl=en From the website: It tests specific device properties you do not regularly see tested by other benchmarks, and runs in a set timeframe. It does produce some "final" scores, but as with every benchmark, you should take those with a grain of salt. It is simply not theoretically possible to produce a single number that accurately describes a device's performance. .. note:: This workload relies on the device being rooted parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` citadel ------- Epic Citadel demo showcasing Unreal Engine 3. The game has very rich graphics details. The workload only moves around its environment for the specified time. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` duration : integer Duration, in seconds, of the run (may need to be adjusted for different devices. default: ``60`` cyclictest ---------- Measures the amount of time that passes between when a timer expires and when the thread which set the timer actually runs. Cyclic test works by taking a time snapshot just prior to waiting for a specific time interval (t1), then taking another time snapshot after the timer finishes (t2), then comparing the theoretical wakeup time with the actual wakeup time (t2 -(t1 + sleep_time)). This value is the latency for that timers wakeup. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. clock : str specify the clock to be used during the test. allowed values: ``'monotonic'``, ``'realtime'`` default: ``'realtime'`` duration : integer Specify the length for the test to run in seconds. default: ``30`` quiet : boolean Run the tests quiet and print only a summary on exit. default: ``True`` thread : integer Set the number of test threads default: ``8`` latency : integer Write the value to /dev/cpu_dma_latency default: ``1000000`` extra_parameters : str Any additional command line parameters to append to the existing parameters above. A list can be found at https://rt.wiki.kernel.org/index.php/Cyclictest or in the help page ``cyclictest -h`` clear_file_cache : boolean Clear file caches before starting test default: ``True`` screen_off : boolean If true it will turn the screen off so that onscreen graphics do not effect the score. This is predominantly for devices without a GPU default: ``True`` dex2oat ------- Benchmarks the execution time of dex2oat (a key part of APK installation process). ART is a new Android runtime in KitKat, which replaces Dalvik VM. ART uses Ahead-Of-Time compilation. It pre-compiles ODEX files used by Dalvik using dex2oat tool as part of APK installation process. This workload benchmarks the time it take to compile an APK using dex2oat, which has a significant impact on the total APK installation time, and therefore user experience. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. instruction_set : str Specifies the instruction set to compile for. Only options supported by the target device can be used. allowed values: ``'arm'``, ``'arm64'``, ``'x86'``, ``'x86_64'``, ``'mips'`` default: ``'arm64'`` dhrystone --------- Runs the Dhrystone benchmark. Original source from:: http://classes.soe.ucsc.edu/cmpe202/benchmarks/standard/dhrystone.c This version has been modified to configure duration and the number of threads used. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. duration : integer The duration, in seconds, for which dhrystone will be executed. Either this or ``mloops`` should be specified but not both. mloops : integer Millions of loops to run. Either this or ``duration`` should be specified, but not both. If neither is specified, this will default to ``100`` threads : integer The number of separate dhrystone "threads" that will be forked. default: ``4`` delay : integer The delay, in seconds, between kicking off of dhrystone threads (if ``threads`` > 1). taskset_mask : integer The processes spawned by the workload will be pinned to cores as specified by this parameter dungeondefenders ---------------- Dungeon Defenders game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` ebizzy ------ ebizzy is designed to generate a workload resembling common web application server workloads. It is highly threaded, has a large in-memory working set with low locality, and allocates and deallocates memory frequently. When running most efficiently, it will max out the CPU. ebizzy description taken from the source code at https://github.com/linux-test-project/ltp/tree/master/utils/benchmark/ebizzy-0.3 parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. threads : integer Number of threads to execute. default: ``2`` seconds : integer Number of seconds. default: ``10`` chunks : integer Number of memory chunks to allocate. default: ``10`` extra_params : str Extra parameters to pass in (e.g. -M to disable mmap). See ebizzy -? for full list of options. facebook -------- Uses com.facebook.patana apk for facebook workload. This workload does the following activities in facebook Login to facebook account. Send a message. Check latest notification. Search particular user account and visit his/her facebook account. Find friends. Update the facebook status .. note:: This workload starts disableUpdate workload as a part of setup to disable online updates, which helps to tackle problem of uncertain behavier during facebook workload run.] parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` geekbench --------- Geekbench provides a comprehensive set of benchmarks engineered to quickly and accurately measure processor and memory performance. http://www.primatelabs.com/geekbench/ From the website: Designed to make benchmarks easy to run and easy to understand, Geekbench takes the guesswork out of producing robust and reliable benchmark results. Geekbench scores are calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are better, with double the score indicating double the performance. The benchmarks fall into one of four categories: - integer performance. - floating point performance. - memory performance. - stream performance. Geekbench benchmarks: http://www.primatelabs.com/geekbench/doc/benchmarks.html Geekbench scoring methedology: http://support.primatelabs.com/kb/geekbench/interpreting-geekbench-scores parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` version : str Specifies which version of the workload should be run. allowed values: ``'2'``, ``'3.0.0'``, ``'3.4.1'``, ``'4.0.1'`` default: ``'4.0.1'`` times : integer Specfies the number of times the benchmark will be run in a "tight loop", i.e. without performaing setup/teardown inbetween. default: ``1`` timeout : integer Timeout for a single iteration of the benchmark. This value is multiplied by ``times`` to calculate the overall run timeout. default: ``900`` disable_update_result : boolean If ``True`` the results file will not be pulled from the devices /data/data/com.primatelabs.geekbench folder. This allows the workload to be run on unrooted devices and the results extracted manually later. geekbench-corporate ------------------- Geekbench provides a comprehensive set of benchmarks engineered to quickly and accurately measure processor and memory performance. http://www.primatelabs.com/geekbench/ From the website: Designed to make benchmarks easy to run and easy to understand, Geekbench takes the guesswork out of producing robust and reliable benchmark results. Geekbench scores are calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are better, with double the score indicating double the performance. The benchmarks fall into one of four categories: - integer performance. - floating point performance. - memory performance. - stream performance. Geekbench benchmarks: http://www.primatelabs.com/geekbench/doc/benchmarks.html Geekbench scoring methedology: http://support.primatelabs.com/kb/geekbench/interpreting-geekbench-scores parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` version : str Specifies which version of the workload should be run. allowed values: ``'4.1.0'`` default: ``'4.1.0'`` times : integer Specfies the number of times the benchmark will be run in a "tight loop", i.e. without performaing setup/teardown inbetween. default: ``1`` timeout : integer Timeout for a single iteration of the benchmark. This value is multiplied by ``times`` to calculate the overall run timeout. default: ``900`` disable_update_result : boolean If ``True`` the results file will not be pulled from the devices /data/data/com.primatelabs.geekbench folder. This allows the workload to be run on unrooted devices and the results extracted manually later. glb_corporate ------------- GFXBench GL (a.k.a. GLBench) v3.0 Corporate version. This is a version of GLBench available through a corporate license (distinct from the version available in Google Play store). parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` times : integer Specifies the number of times the benchmark will be run in a "tight loop", i.e. without performaing setup/teardown inbetween. constraint: ``value > 0`` default: ``1`` resolution : str Explicitly specifies the resultion under which the benchmark will be run. If not specfied, device's native resoution will used. allowed values: ``'720p'``, ``'1080p'``, ``'720'``, ``'1080'`` test_id : str ID of the GFXBench test to be run. allowed values: ``'gl_alu'``, ``'gl_alu_off'``, ``'gl_blending'``, ``'gl_blending_off'``, ``'gl_driver'``, ``'gl_driver_off'``, ``'gl_fill'``, ``'gl_fill_off'``, ``'gl_manhattan'``, ``'gl_manhattan_off'``, ``'gl_trex'``, ``'gl_trex_battery'``, ``'gl_trex_off'``, ``'gl_trex_qmatch'``, ``'gl_trex_qmatch_highp'`` default: ``'gl_manhattan_off'`` run_timeout : integer Time out for workload execution. The workload will be killed if it hasn't completed withint this period. default: ``600`` glbenchmark ----------- Measures the graphics performance of Android devices by testing the underlying OpenGL (ES) implementation. http://gfxbench.com/about-gfxbench.jsp From the website: The benchmark includes console-quality high-level 3D animations (T-Rex HD and Egypt HD) and low-level graphics measurements. With high vertex count and complex effects such as motion blur, parallax mapping and particle systems, the engine of GFXBench stresses GPUs in order provide users a realistic feedback on their device. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` version : str Specifies which version of the benchmark to run (different versions support different use cases). allowed values: ``'2.7'``, ``'2.5'`` default: ``'2.7'`` use_case : str Specifies which usecase to run, as listed in the benchmark menu; e.g. ``'GLBenchmark 2.5 Egypt HD'``. For convenience, two aliases are provided for the most common use cases: ``'egypt'`` and ``'t-rex'``. These could be use instead of the full use case title. For version ``'2.7'`` it defaults to ``'t-rex'``, for version ``'2.5'`` it defaults to ``'egypt-classic'``. variant : str Specifies which variant of the use case to run, as listed in the benchmarks menu (small text underneath the use case name); e.g. ``'C24Z16 Onscreen Auto'``. For convenience, two aliases are provided for the most common variants: ``'onscreen'`` and ``'offscreen'``. These may be used instead of full variant names. default: ``'onscreen'`` times : integer Specfies the number of times the benchmark will be run in a "tight loop", i.e. without performaing setup/teardown inbetween. default: ``1`` timeout : integer Specifies how long, in seconds, UI automation will wait for results screen to appear before assuming something went wrong. default: ``200`` gmail ----- A workload to perform standard productivity tasks within Gmail. The workload carries out various tasks, such as creating new emails, attaching images and sending them. Test description: 1. Open Gmail application 2. Click to create New mail 3. Attach an image from the local images folder to the email 4. Enter recipient details in the To field 5. Enter text in the Subject field 6. Enter text in the Compose field 7. Click the Send mail button Known working APK version: 7.6.18.160170480 parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path recipient : str The email address of the recipient. Setting a void address will stop any mesage failures clogging up your device inbox default: ``'wa-devnull@mailinator.com'`` test_image : str An image to be copied onto the device that will be attached to the email default: ``'uxperf_1600x1200.jpg'`` googlemap --------- Navigation app. Stock map provided by Google Inc. Based on revent, we can use this workload to do multiple tasks such as navigation usecases, swipe & pinch etc. Provided revent is for Odriod XU3 for navigation use case. For running on other devices, we need to build revent. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` googlephotos ------------ A workload to perform standard productivity tasks with Google Photos. The workload carries out various tasks, such as browsing images, performing zooms, and post-processing the image. Test description: 1. Four images are copied to the device 2. The application is started in offline access mode 3. Gestures are performed to pinch zoom in and out of the selected image 4. The colour of a selected image is edited by selecting the colour menu, incrementing the colour, resetting the colour and decrementing the colour using the seek bar. 5. A crop test is performed on a selected image. UiAutomator does not allow the selection of the crop markers so the image is tilted positively, reset and then tilted negatively to get a similar cropping effect. 6. A rotate test is performed on a selected image, rotating anticlockwise 90 degrees, 180 degrees and 270 degrees. Known working APK version: 1.21.0.123444480 parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path test_images : list_of_strs A list of four JPEG and/or PNG files to be pushed to the device. Absolute file paths may be used but tilde expansion must be escaped. constraint: ``len(unique(value)) == 4`` default: ``['uxperf_1200x1600.png', 'uxperf_1600x1200.jpg', 'uxperf_2448x3264.png', 'uxperf_3264x2448.jpg']`` googleplaybooks --------------- A workload to perform standard productivity tasks with googleplaybooks. This workload performs various tasks, such as searching for a book title online, browsing through a book, adding and removing notes, word searching, and querying information about the book. Test description: 1. Open Google Play Books application 2. Dismisses sync operation (if applicable) 3. Searches for a book title 4. Adds books to library if not already present 5. Opens 'My Library' contents 6. Opens selected book 7. Gestures are performed to swipe between pages and pinch zoom in and out of a page 8. Selects a specified chapter based on page number from the navigation view 9. Selects a word in the centre of screen and adds a test note to the page 10. Removes the test note from the page (clean up) 11. Searches for the number of occurrences of a common word throughout the book 12. Switches page styles from 'Day' to 'Night' to 'Sepia' and back to 'Day' 13. Uses the 'About this book' facility on the currently selected book NOTE: This workload requires a network connection (ideally, wifi) to run, a Google account to be setup on the device, and payment details for the account. Free books require payment details to have been setup otherwise it fails. Tip: Install the 'Google Opinion Rewards' app to bypass the need to enter valid card/bank detail. Known working APK version: 3.13.17 parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path search_book_title : str The book title to search for within Google Play Books archive. The book must either be already in the account's library, or free to purchase. default: ``'Nikola Tesla: Imagination and the Man That Invented the 20th Century'`` library_book_title : str The book title to search for within My Library. The Library name can differ (usually shorter) to the Store name. If left blank, the ``search_book_title`` will be used. default: ``'Nikola Tesla'`` select_chapter_page_number : integer The Page Number to search for within a selected book's Chapter list. Note: Accepts integers only. default: ``4`` search_word : str The word to search for within a selected book. Note: Accepts single words only. default: ``'the'`` account : str If you are running this workload on a device which has more than one Google account setup, then this parameter is used to select which account to select when prompted. The account requires the book to have already been purchased or payment details already associated with the account. If omitted, the first account in the list will be selected if prompted. googleslides ------------ A workload to perform standard productivity tasks with Google Slides. The workload carries out various tasks, such as creating a new presentation, adding text, images, and shapes, as well as basic editing and playing a slideshow. This workload should be able to run without a network connection. There are two main scenarios: 1. create test: a presentation is created in-app and some editing done on it, 2. load test: a pre-existing PowerPoint file is copied onto the device for testing. --- create --- Create a new file in the application and perform basic editing on it. This test also requires an image file specified by the param ``test_image`` to be copied onto the device. Test description: 1. Start the app and skip the welcome screen. Dismiss the work offline banner if present. 2. Go to the app settings page and enables PowerPoint compatibility mode. This allows PowerPoint files to be created inside Google Slides. 3. Create a new PowerPoint presentation in the app (PPT compatibility mode) with a title slide and save it to device storage. 4. Insert another slide and to it insert the pushed image by picking it from the gallery. 5. Insert a final slide and add a shape to it. Resize and drag the shape to modify it. 6. Finally, navigate back to the documents list. --- load --- Copy a PowerPoint presentation onto the device to test slide navigation. The PowerPoint file to be copied is given by ``test_file``. Test description: 1. From the documents list (following the create test), open the specified PowerPoint by navigating into device storage and wait for it to be loaded. 2. A navigation test is performed while the file is in editing mode (i.e. not slideshow). swiping forward to the next slide until ``slide_count`` swipes are performed. 3. While still in editing mode, the same action is done in the reverse direction back to the first slide. 4. Enter presentation mode by selecting to play the slideshow. 5. Swipe forward to play the slideshow, for a maximum number of ``slide_count`` swipes. 6. Finally, repeat the previous step in the reverse direction while still in presentation mode, navigating back to the first slide. NOTE: There are known issues with the reliability of this workload on some targets. It MAY NOT ALWAYS WORK on your device. If you do run into problems, it might help to set ``do_text_entry`` parameter to ``False``. Known working APK version: 1.7.032.06 parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path test_image : str An image to be copied onto the device that will be embedded in the PowerPoint file as part of the test. default: ``'uxperf_1600x1200.jpg'`` test_file : str If specified, the workload will copy the PowerPoint file to be used for testing onto the device. Otherwise, a file will be created inside the app. default: ``'uxperf_test_doc.pptx'`` slide_count : integer Number of slides in aforementioned local file. Determines number of swipe actions when playing slide show. default: ``5`` do_text_entry : boolean If set to ``True``, will attempt to enter text in the first slide as part of the test. Currently seems to be problematic on some devices, most notably Samsung devices. default: ``True`` gunbros2 -------- Gun Bros. 2 game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` hackbench --------- Hackbench runs a series of tests for the Linux scheduler. For details, go to: https://github.com/linux-test-project/ltp/ parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. datasize : integer Message size in bytes. default: ``100`` groups : integer Number of groups. default: ``10`` loops : integer Number of loops. default: ``100`` fds : integer Number of file descriptors. default: ``40`` extra_params : str Extra parameters to pass in. See the hackbench man page or type `hackbench --help` for list of options. duration : integer Test duration in seconds. default: ``30`` homescreen ---------- A workload that goes to the home screen and idles for the the specified duration. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. duration : integer Specifies the duration, in seconds, of this workload. default: ``20`` hwuitest -------- Tests UI rendering latency on android devices parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. test : caseless_string The test to run: - ``'shadowgrid'``: creates a grid of rounded rects that cast shadows, high CPU & GPU load - ``'rectgrid'``: creates a grid of 1x1 rects - ``'oval'``: draws 1 oval allowed values: ``'shadowgrid'``, ``'rectgrid'``, ``'oval'`` default: ``'shadowgrid'`` loops : integer The number of test iterations. default: ``3`` frames : integer The number of frames to run the test over. default: ``150`` idle ---- Do nothing for the specified duration. On android devices, this may optionally stop the Android run time, if ``stop_android`` is set to ``True``. .. note:: This workload requires the device to be rooted. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. duration : integer Specifies the duration, in seconds, of this workload. default: ``20`` stop_android : boolean Specifies whether the Android run time should be stopped. (Can be set only for Android devices). iozone ------ Iozone is a filesystem benchmark that runs a series of disk I/O performance tests. Here is a list of tests that you can run in the iozone workload. The descriptions are from the official iozone document. 0 - Write Test Measure performance of writing a new file. Other tests rely on the file written by this, so it must always be enabled (WA will automatically neable this if not specified). 1 - Rewrite Test Measure performance of writing an existing file. 2 - Read Test Measure performance of reading an existing file. 3 - Reread Test Measure performance of rereading an existing file. 4 - Random Read Test Measure performance of reading a file by accessing random locations within the file. 5 - Random Write Test Measure performance of writing a file by accessing random locations within the file. 6 - Backwards Read Test Measure performance of reading a file backwards. 7 - Record Rewrite Test Measure performance of writing and rewriting a particular spot within the file. 8 - Strided Read Test Measure performance of reading a file with strided access behavior. 9 - Fwrite Test Measure performance of writing a file using the library function fwrite() that performances buffered write operations. 10 - Frewrite Test Measure performance of writing a file using the the library function fwrite() that performs buffered and blocked write operations. 11 - Fread Test Measure performance of reading a file using the library function fread() that performs buffered and blocked read operations. 12 - Freread Test Same as the Fread Test except the current file being read was read previously sometime in the past. By default, iozone will run all tests in auto mode. To run specific tests, they must be written in the form of: [0,1,4,5] Please enable classifiers in your agenda or config file in order to display the results properly in the results.csv file. The official website for iozone is at www.iozone.org. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. tests : list_of_ints List of performance tests to run. allowed values: ``0``, ``1``, ``2``, ``3``, ``4``, ``5``, ``6``, ``7``, ``8``, ``9``, ``10``, ``11``, ``12`` auto_mode : boolean Run tests in auto mode. default: ``True`` timeout : integer Timeout for the workload. default: ``14400`` file_size : integer Fixed file size. record_length : integer Fixed record length. threads : integer Number of threads other_params : str Other parameter. Run iozone -h to see list of options. ironman3 -------- Iron Man 3 game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` krazykart --------- Krazy Kart Racing game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` linpack ------- The LINPACK Benchmarks are a measure of a system's floating point computing power. http://en.wikipedia.org/wiki/LINPACK_benchmarks From the article: Introduced by Jack Dongarra, they measure how fast a computer solves a dense n by n system of linear equations Ax = b, which is a common task in engineering. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` output_file : str On-device output file path. linpack-cli ----------- linpack benchmark with a command line interface Benchmarks FLOPS (floating point operations per second). This is the oldschool version of the bencmark. Source may be viewed here: http://www.netlib.org/benchmark/linpackc.new parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. array_size : integer size of arrays to be used by the benchmark. default: ``200`` lmbench ------- Run a subtest from lmbench, a suite of portable ANSI/C microbenchmarks for UNIX/POSIX. In general, lmbench measures two key features: latency and bandwidth. This workload supports a subset of lmbench tests. lat_mem_rd can be used to measure latencies to memory (including caches). bw_mem can be used to measure bandwidth to/from memory over a range of operations. Further details, and source code are available from: http://sourceforge.net/projects/lmbench/. See lmbench/bin/README for license details. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. test : str Specifies an lmbench test to run. allowed values: ``'lat_mem_rd'``, ``'bw_mem'`` default: ``'lat_mem_rd'`` stride : list_or_type Stride for lat_mem_rd test. Workload will iterate over one or more integer values. default: ``[128]`` thrash : boolean Sets -t flag for lat_mem_rd_test default: ``True`` size : list_or_string Data set size for lat_mem_rd bw_mem tests. default: ``'4m'`` mem_category : list_or_string List of memory catetories for bw_mem test. default: ``('rd', 'wr', 'cp', 'frd', 'fwr', 'fcp', 'bzero', 'bcopy')`` parallelism : integer Parallelism flag for tests that accept it. warmup : integer Warmup flag for tests that accept it. repetitions : integer Repetitions flag for tests that accept it. force_abi : str Override device abi with this value. Can be used to force arm32 on 64-bit devices. run_timeout : integer Timeout for execution of the test. default: ``900`` times : integer Specifies the number of times the benchmark will be run in a "tight loop", i.e. without performaing setup/teardown inbetween. This parameter is distinct from "repetitions", as the latter takes place within the benchmark and produces a single result. constraint: ``value > 0`` default: ``1`` taskset_mask : integer Specifies the CPU mask the benchmark process will be pinned to. manual ------ Yields control to the user, either for a fixed period or based on user input, to perform custom operations on the device, about which workload automation does not know of. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. duration : integer Control of the devices is yielded for the duration (in seconds) specified. If not specified, ``user_triggered`` is assumed. user_triggered : boolean If ``True``, WA will wait for user input after starting the workload; otherwise fixed duration is expected. Defaults to ``True`` if ``duration`` is not specified, and ``False`` otherwise. view : str Specifies the View of the workload. This enables instruments that require a View to be specified, such as the ``fps`` instrument. This is required for using "SurfaceFlinger" to collect FPS statistics and is primarily used on devices pre API level 23 default: ``'SurfaceView'`` package : str Specifies the package name of the workload. This enables instruments that require a Package to be specified, such as the ``fps`` instrument. This allows for "gfxinfo" to be used and is the preferred method of collection for FPS statistics on devices API level 23+ enable_logcat : boolean If ``True``, ``manual`` workload will collect logcat as part of the results. memcpy ------ Runs memcpy in a loop. This will run memcpy in a loop for a specified number of times on a buffer of a specified size. Additionally, the affinity of the test can be set to one or more specific cores. This workload is single-threaded. It genrates no scores or metrics by itself. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. buffer_size : integer Specifies the size, in bytes, of the buffer to be copied. default: ``5242880`` iterations : integer Specfies the number of iterations that will be performed. default: ``1000`` cpus : list A list of integers specifying ordinals of cores to which the affinity of the test process should be set. If not specified, all avaiable cores will be used. nenamark -------- NenaMark is an OpenGL-ES 2.0 graphics performance benchmark for Android devices. http://nena.se/nenamark_story From the website: The NenaMark2 benchmark scene averages about 45k triangles, with a span between 26k and 68k triangles. It averages 96 batches per frame and contains about 15 Mb of texture data (non-packed). parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` duration : integer Number of seconds to wait before considering the benchmark finished default: ``120`` octaned8 -------- Runs the Octane d8 benchmark. This workload runs d8 binaries built from source and placed in the dependencies folder along with test assets from https://github.com/chromium/octane which also need to be placed in an assets folder within the dependencies folder. Original source from:: https://github.com/v8/v8/wiki/D8%20on%20Android parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. run_timeout : integer Timeout, in seconds, for the test execution. default: ``180`` peacekeeper ----------- Peacekeeper is a free and fast browser test that measures a browser's speed. .. note:: This workload requires a network connection as well as support for one of the two currently-supported browsers. Moreover, TC2 has compatibility issue with chrome parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` browser : str The browser to be benchmarked. allowed values: ``'firefox'``, ``'chrome'`` default: ``'firefox'`` output_file : str The result URL of peacekeeper benchmark will be written into this file on device after completion of peacekeeper benchmark. Defaults to peacekeeper.txt in the device's ``working_directory``. peacekeeper_url : str The URL to run the peacekeeper benchmark. default: ``'http://peacekeeper.futuremark.com/run.action'`` power_loadtest -------------- power_LoadTest (part of ChromeOS autotest suite) continuously cycles through a set of browser-based activities and monitors battery drain on a device. .. note:: This workload *must* be run inside a CromeOS SDK chroot. See: https://www.chromium.org/chromium-os/testing/power-testing parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. board : str The name of the board to be used for the test. If this is not specified, BOARD environment variable will be used. variant : str The variant of the test to run; If not specified, the full power_LoadTest will run (until the device battery is drained). The only other variant available in the vanilla test is "1hour", but further variants may be added by providing custom control files. test_that_args : arguments Extra arguments to be passed to test_that_invocation. run_timeout : integer Timeout, in seconds, for the test execution. default: ``86400`` quadrant -------- Quadrant is a benchmark for mobile devices, capable of measuring CPU, memory, I/O and 3D graphics performance. http://www.aurorasoftworks.com/products/quadrant From the website: Quadrant outputs a score for the following categories: 2D, 3D, Mem, I/O, CPU , Total. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` real-linpack ------------ This version of `Linpack ` was developed by Dave Butcher. RealLinpack tries to find the number of threads that give you the maximum linpack score. RealLinpack runs 20 runs of linpack for each number of threads and calculates the mean and confidence. It stops when the score's confidence interval drops below the current best score interval. That is, when (current_score + confidence) < (best_score - best_score_confidence) parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` max_threads : integer The maximum number of threads that real linpack will try. constraint: ``value > 0`` default: ``16`` realracing3 ----------- Real Racing 3 game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` recentfling ----------- Tests UI jank on android devices. For this workload to work, ``recentfling.sh`` and ``defs.sh`` must be placed in ``~/.workload_automation/dependencies/recentfling/``. These can be found in the `AOSP Git repository `_. To change the apps that are opened at the start of the workload you will need to modify the ``defs.sh`` file. You will need to add your app to ``dfltAppList`` and then add a variable called ``{app_name}Activity`` with the name of the activity to launch (where ``{add_name}`` is the name you put into ``dfltAppList``). You can get a list of activities available on your device by running ``adb shell pm list packages -f`` parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. loops : integer The number of test iterations. default: ``3`` start_apps : boolean If set to ``False``,no apps will be started before flinging through the recent apps list (in which the assumption is there are already recently started apps in the list. default: ``True`` device_name : str If set, recentfling will use the fling parameters for this device instead of automatically guessing the device. This can also be used if the device is not supported by recentfling, but its screensize is similar to that of one that is supported. For possible values, check your recentfling.sh. At the time of writing, valid values are: 'shamu', 'hammerhead', 'angler', 'ariel', 'mtp8996', 'bullhead' or 'volantis'. rt-app ------ A test application that simulates configurable real-time periodic load. rt-app is a test application that starts multiple periodic threads in order to simulate a real-time periodic load. It supports SCHED_OTHER, SCHED_FIFO, SCHED_RR as well as the AQuoSA framework and SCHED_DEADLINE. The load is described using JSON-like config files. Below are a couple of simple examples. Simple use case which creates a thread that run 1ms then sleep 9ms until the use case is stopped with Ctrl+C: .. code-block:: json { "tasks" : { "thread0" : { "loop" : -1, "run" : 20000, "sleep" : 80000 } }, "global" : { "duration" : 2, "calibration" : "CPU0", "default_policy" : "SCHED_OTHER", "pi_enabled" : false, "lock_pages" : false, "logdir" : "./", "log_basename" : "rt-app1", "ftrace" : false, "gnuplot" : true, } } Simple use case with 2 threads that runs for 10 ms and wake up each other until the use case is stopped with Ctrl+C .. code-block:: json { "tasks" : { "thread0" : { "loop" : -1, "run" : 10000, "resume" : "thread1", "suspend" : "thread0" }, "thread1" : { "loop" : -1, "run" : 10000, "resume" : "thread0", "suspend" : "thread1" } } } Please refer to the existing configs in ``$WA_ROOT/wlauto/workloads/rt_app/use_case`` for more examples. The version of rt-app currently used with this workload contains enhancements and modifications done by Linaro. The source code for this version may be obtained here: http://git.linaro.org/power/rt-app.git The upstream version of rt-app is hosted here: https://github.com/scheduler-tools/rt-app parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. config : str Use case configuration file to run with rt-app. This may be either the name of one of the "standard" configurations included with the workload. or a path to a custom JSON file provided by the user. Either way, the ".json" extension is implied and will be added automatically if not specified in the argument. The following is the list of standard configurations currently included with the workload: browser-long.json, spreading-tasks.json, mp3-short.json, camera-short.json, browser-short.json, mp3-long.json, video-long.json, camera-long.json, taskset.json, video-short.json default: ``'taskset'`` duration : integer Duration of the workload execution in Seconds. If specified, this will override the corresponding parameter in the JSON config. taskset_mask : integer Constrain execution to specific CPUs. uninstall_on_exit : boolean If set to ``True``, rt-app binary will be uninstalled from the device at the end of the run. force_install : boolean If set to ``True``, rt-app binary will always be deployed to the target device at the beginning of the run, regardless of whether it was already installed there. shellscript ----------- Runs an arbitrary shellscript on the device. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. script_file : str (mandatory) The path (on the host) to the shell script file. This must be an absolute path (though it may contain ~). argstring : str A string that should contain arguments passed to the script. timeout : integer Timeout, in seconds, for the script run time. default: ``60`` skype ----- A workload to perform standard productivity tasks within Skype. The workload logs in to the Skype application, selects a recipient from the contacts list and then initiates either a voice or video call. Test description: 1. Open Skype application 2. Log in to a pre-defined account 3. Select a recipient from the Contacts list 4. Initiate either a ``voice`` or ``video`` call for ``duration`` time (in seconds) Note: The actual duration of the call may not match exactly the intended duration due to the uiautomation overhead. **Skype Setup** - You must have a Skype account set up and its credentials passed as parameters into this workload - The contact to be called must be added (and has accepted) to the account. It's possible to have multiple contacts in the list, however the contact to be called *must* be visible on initial navigation to the list. - For video calls the contact must be able to received the call. This means that there must be a Skype client running (somewhere) with the contact logged in and that client must have been configured to auto-accept calls from the account on the device (how to set this varies between different versions of Skype and between platforms -- please search online for specific instructions). https://support.skype.com/en/faq/FA3751/can-i-automatically-answer-all-my-calls-with-video-in-skype-for-windows-desktop Known working APK version: 7.01.0.669 parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path login_name : str (mandatory) Account to use when logging into the device from which the call will be made login_pass : str (mandatory) Password associated with the account to log into the device contact_name : str This is the contact display name as it appears in the people list default: ``'Echo / Sound Test Service'`` duration : integer This is the target duration of the call in seconds default: ``10`` action : str Action to take - either voice call (default) or video allowed values: ``'voice'``, ``'video'`` default: ``'voice'`` smartbench ---------- Smartbench is a multi-core friendly benchmark application that measures the overall performance of an android device. It reports both Productivity and Gaming Index. https://play.google.com/store/apps/details?id=com.smartbench.twelve&hl=en From the website: It will be better prepared for the quad-core world. Unfortunately this also means it will run slower on older devices. It will also run slower on high-resolution tablet devices. All 3D tests are now rendered in full native resolutions so naturally it will stress hardware harder on these devices. This also applies to higher resolution hand-held devices. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` spec2000 -------- SPEC2000 benchmarks measuring processor, memory and compiler. http://www.spec.org/cpu2000/ From the web site: SPEC CPU2000 is the next-generation industry-standardized CPU-intensive benchmark suite. SPEC designed CPU2000 to provide a comparative measure of compute intensive performance across the widest practical range of hardware. The implementation resulted in source code benchmarks developed from real user applications. These benchmarks measure the performance of the processor, memory and compiler on the tested system. .. note:: At the moment, this workload relies on pre-built SPEC binaries (included in an asset bundle). These binaries *must* be built according to rules outlined here:: http://www.spec.org/cpu2000/docs/runrules.html#toc_2.0 in order for the results to be valid SPEC2000 results. .. note:: This workload does not attempt to generate results in an admissible SPEC format. No metadata is provided (though some, but not all, of the required metdata is collected by WA elsewhere). It is upto the user to post-process results to generated SPEC-admissible results file, if that is their intention. *base vs peak* SPEC2000 defines two build/test configuration: base and peak. Base is supposed to use basic configuration (e.g. default compiler flags) with no tuning, and peak is specifically optimized for a system. Since this workload uses externally-built binaries, there is no way for WA to be sure what configuration is used -- the user is expected to keep track of that. Be aware that base/peak also come with specific requirements for the way workloads are run (e.g. how many instances on multi-core systems):: http://www.spec.org/cpu2000/docs/runrules.html#toc_3 These are not enforced by WA, so it is again up to the user to ensure that correct workload parameters are specfied inthe agenda, if they intend to collect "official" SPEC results. (Those interested in collecting official SPEC results should also note that setting runtime parameters would violate SPEC runs rules that state that no configuration must be done to the platform after boot). *bundle structure* This workload expects the actual benchmark binaries to be provided in a tarball "bundle" that has a very specific structure. At the top level of the tarball, there should be two directories: "fp" and "int" -- for each of the SPEC2000 categories. Under those, there is a sub-directory per benchmark. Each benchmark sub-directory contains three sub-sub-directorie: - "cpus" contains a subdirectory for each supported cpu (e.g. a15) with a single executable binary for that cpu, in addition to a "generic" subdirectory that has not been optimized for a specific cpu and should run on any ARM system. - "data" contains all additional files (input, configuration, etc) that the benchmark executable relies on. - "scripts" contains one or more one-liner shell scripts that invoke the benchmark binary with appropriate command line parameters. The name of the script must be in the format [.].sh, i.e. name of benchmark, optionally followed by variant name, followed by ".sh" extension. If there is more than one script, then all of them must have a variant; if there is only one script the it should not contain a variant. A typical bundle may look like this:: |- fp | |-- ammp | | |-- cpus | | | |-- generic | | | | |-- ammp | | | |-- a15 | | | | |-- ammp | | | |-- a7 | | | | |-- ammp | | |-- data | | | |-- ammp.in | | |-- scripts | | | |-- ammp.sh | |-- applu . . . . . . . . . |- int . parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. benchmarks : list_or_string Specifies the SPEC benchmarks to run. mode : str SPEC benchmarks can report either speed to execute or throughput/rate. In the latter case, several "threads" will be spawned. allowed values: ``'speed'``, ``'rate'`` default: ``'speed'`` number_of_threads : integer Specify the number of "threads" to be used in 'rate' mode. (Note: on big.LITTLE systems this is the number of threads, for *each cluster*). force_extract_assets : boolean if set to ``True``, will extract assets from the bundle, even if they are already extracted. Note: this option implies ``force_push_assets``. force_push_assets : boolean If set to ``True``, assets will be pushed to device even if they're already present. timeout : integer Timeout, in seconds, for the execution of single spec test. default: ``1200`` sqlitebm -------- Measures the performance of the sqlite database. It determines within what time the target device processes a number of SQL queries. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` stream ------ Measures memory bandwidth. The original source code be found on: https://www.cs.virginia.edu/stream/FTP/Code/ parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. threads : integer The number of threads to execute if OpenMP is enabled stress_ng --------- stress-ng will stress test a computer system in various selectable ways. It was designed to exercise various physical subsystems of a computer as well as the various operating system kernel interfaces. stress-ng can also measure test throughput rates; this can be useful to observe performance changes across different operating system releases or types of hardware. However, it has never been intended to be used as a precise benchmark test suite, so do NOT use it in this manner. The official website for stress-ng is at: http://kernel.ubuntu.com/~cking/stress-ng/ Source code are available from: http://kernel.ubuntu.com/git/cking/stress-ng.git/ parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. stressor : str Stress test case name. The cases listed in allowed values come from the stable release version 0.01.32. The binary included here compiled from dev version 0.06.01. Refer to man page for the definition of each stressor. allowed values: ``'cpu'``, ``'io'``, ``'fork'``, ``'switch'``, ``'vm'``, ``'pipe'``, ``'yield'``, ``'hdd'``, ``'cache'``, ``'sock'``, ``'fallocate'``, ``'flock'``, ``'affinity'``, ``'timer'``, ``'dentry'``, ``'urandom'``, ``'sem'``, ``'open'``, ``'sigq'``, ``'poll'`` default: ``'cpu'`` threads : integer The number of workers to run. Specifying a negative or zero value will select the number of online processors. duration : integer Timeout for test execution in seconds default: ``60`` sysbench -------- SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load. The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all. **Features of SysBench** * file I/O performance * scheduler performance * memory allocation and transfer speed * POSIX threads implementation performance * database server performance See: https://github.com/akopytov/sysbench parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. timeout : integer timeout for workload execution (adjust from default if running on a slow device and/or specifying a large value for ``max_requests`` default: ``300`` test : str sysbench test to run allowed values: ``'fileio'``, ``'cpu'``, ``'memory'``, ``'threads'``, ``'mutex'`` default: ``'cpu'`` threads : integer The number of threads sysbench will launch default: ``8`` num_threads : integer The number of threads sysbench will launch, overrides ``threads`` (old parameter name) max_requests : integer The limit for the total number of requests. max_time : integer The limit for the total execution time. If neither this nor ``max_requests`` is specified, this will default to 30 seconds. file_test_mode : str File test mode to use. This should only be specified if ``test`` is ``"fileio"``; if that is the case and ``file_test_mode`` is not specified, it will default to ``"seqwr"`` (please see sysbench documentation for explanation of various modes). allowed values: ``'seqwr'``, ``'seqrewr'``, ``'seqrd'``, ``'rndrd'``, ``'rndwr'``, ``'rndrw'`` cmd_params : str Additional parameters to be passed to sysbench as a single stiring force_install : boolean Always install binary found on the host, even if already installed on device default: ``True`` taskset_mask : integer The processes spawned by sysbench will be pinned to cores as specified by this parameter telemetry --------- Executes Google's Telemetery benchmarking framework Url: https://www.chromium.org/developers/telemetry From the web site: Telemetry is Chrome's performance testing framework. It allows you to perform arbitrary actions on a set of web pages and report metrics about it. The framework abstracts: - Launching a browser with arbitrary flags on any platform. - Opening a tab and navigating to the page under test. - Fetching data via the Inspector timeline and traces. - Using Web Page Replay to cache real-world websites so they don't change when used in benchmarks. Design Principles - Write one performance test that runs on all platforms - Windows, Mac, Linux, Chrome OS, and Android for both Chrome and ContentShell. - Runs on browser binaries, without a full Chromium checkout, and without having to build the browser yourself. - Use WebPageReplay to get repeatable test results. - Clean architecture for writing benchmarks that keeps measurements and use cases separate. - Run on non-Chrome browsers for comparative studies. This instrument runs telemetry via its ``run_benchmark`` script (which must be in PATH or specified using ``run_benchmark_path`` parameter) and parses metrics from the resulting output. **device setup** The device setup will depend on whether you're running a test image (in which case little or no setup should be necessary) parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. run_benchmark_path : str This is the path to run_benchmark script which runs a Telemetry benchmark. If not specified, WA will look for Telemetry in its dependencies; if not found there, Telemetry will be downloaded. test : str Specifies the telemetry test to run. default: ``'page_cycler.top_10_mobile'`` run_benchmark_params : str Additional paramters to be passed to ``run_benchmark``. run_timeout : integer Timeout for execution of the test. default: ``900`` extract_fps : boolean if ``True``, FPS for the run will be computed from the trace (must be enabled). target_config : str Manually specify target configuration for telemetry. This must contain --browser option plus any addition options Telemetry requires for a particular target (e.g. --device or --remote) templerun --------- Templerun game. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` thechase -------- The Chase demo showcasing the capabilities of Unity game engine. This demo, is a static video-like game demo, that demonstrates advanced features of the unity game engine. It loops continuously until terminated. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` duration : integer Duration, in seconds, note that the demo loops the same (roughly) 60 second sceene until stopped. default: ``70`` truckerparking3d ---------------- Trucker Parking 3D game. (yes, apparently that's a thing...) parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``500`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` idle_time : integer The time you wish the device to remain idle for (if a value is given then this overrides any .run revent file). check_states : boolean Use visual state detection to verify the state of the workload after setup and run assets_push_timeout : integer Timeout used during deployment of the assets package (if there is one). default: ``500`` vellamo ------- Android benchmark designed by Qualcomm. Vellamo began as a mobile web benchmarking tool that today has expanded to include three primary chapters. The Browser Chapter evaluates mobile web browser performance, the Multicore chapter measures the synergy of multiple CPU cores, and the Metal Chapter measures the CPU subsystem performance of mobile processors. Through click-and-go test suites, organized by chapter, Vellamo is designed to evaluate: UX, 3D graphics, and memory read/write and peak bandwidth performance, and much more! Note: Vellamo v3.0 fails to run on Juno parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` version : str Specify the version of Vellamo to be run. If not specified, the latest available version will be used. allowed values: ``'3.2.4'``, ``'2.0.3'``, ``'3.0'`` default: ``'3.2.4'`` benchmarks : list_of_strs Specify which benchmark sections of Vellamo to be run. Only valid on version 3.0 and newer. NOTE: Browser benchmark can be problematic and seem to hang,just wait and it will progress after ~5 minutes allowed values: ``'Browser'``, ``'Metal'``, ``'Multi'`` default: ``['Browser', 'Metal', 'Multi']`` browser : integer Specify which of the installed browsers will be used for the tests. The number refers to the order in which browsers are listed by Vellamo. E.g. ``1`` will select the first browser listed, ``2`` -- the second, etc. Only valid for version ``3.0``. default: ``1`` video ----- Plays a video file using the standard android video player for a predetermined duration. The video can be specified either using ``resolution`` workload parameter, in which case `Big Buck Bunny`_ MP4 video of that resolution will be downloaded and used, or using ``filename`` parameter, in which case the video file specified will be used. .. _Big Buck Bunny: http://www.bigbuckbunny.org/ parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. play_duration : integer Playback duration of the video file. This become the duration of the workload. default: ``20`` resolution : str Specifies which resolution video file to play. allowed values: ``'480p'``, ``'720p'``, ``'1080p'`` default: ``'720p'`` filename : str The name of the video file to play. This can be either a path to the file anywhere on your file system, or it could be just a name, in which case, the workload will look for it in ``~/.workloads_automation/dependency/video`` *Note*: either resolution or filename should be specified, but not both! force_dependency_push : boolean If true, video will always be pushed to device, regardless of whether the file is already on the device. Default is ``False``. videostreaming -------------- Uses the FREEdi video player to search, stream and play the specified video content from YouTube. parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` video_name : str Name of the video to be played. resolution : str Resolution of the video to be played. If video_name is setthis setting will be ignored allowed values: ``'320p'``, ``'720p'``, ``'1080p'`` default: ``'320p'`` sampling_interval : integer Time interval, in seconds, after which the status of the video playback to be monitoreThe elapsed time of the video playback is monitored after after every ``sampling_interval`` seconds and compared against the actual time elapsed and the previous sampling point. If the video elapsed time is less that (sampling time - ``tolerance``) , then the playback is aborted as the video has not been playing continuously. default: ``20`` tolerance : integer Specifies the amount, in seconds, by which sampling time is allowed to deviate from elapsed video playback time. If the delta is greater than this value (which could happen due to poor network connection), workload result will be invalidated. default: ``3`` run_timeout : integer The duration in second for which to play the video default: ``200`` youtube ------- A workload to perform standard productivity tasks within YouTube. The workload plays a video from the app, determined by the ``video_source`` parameter. While the video is playing, a some common actions are done such as video seeking, pausing playback and navigating the comments section. Test description: The ``video_source`` parameter determines where the video to be played will be found in the app. Possible values are ``search``, ``home``, ``my_videos``, and ``trending``. -A. search - Goes to the search view, does a search for the given term, and plays the first video in the results. The parameter ``search_term`` must also be provided in the agenda for this to work. This is the default mode. -B. home - Scrolls down once on the app's home page to avoid ads (if present, would be first video), then select and plays the video that appears at the top of the list. -C. my_videos - Goes to the 'My Videos' section of the user's account page and plays a video from there. The user must have at least one uploaded video for this to work. -D. trending - Goes to the 'Trending Videos' section of the app, and plays the first video in the trending videos list. For the selected video source, the following test steps are performed: 1. Navigate to the general app settings page to disable autoplay. This improves test stability and predictability by preventing screen transition to load a new video while in the middle of the test. 2. Select the video from the source specified above, and dismiss any potential embedded advert that may pop-up before the actual video. 3. Let the video play for a few seconds, pause it, then resume. 4. Expand the info card that shows video metadata, then collapse it again. 5. Scroll down to the end of related videos and comments under the info card, and then back up to the start. A maximum of 5 swipe actions is performed in either direction. Known working APK version: 11.19.56 parameters ~~~~~~~~~~ modules : list Lists the modules to be loaded by this extension. A module is a plug-in that further extends functionality of an extension. install_timeout : integer Timeout for the installation of the apk. default: ``300`` check_apk : boolean When set to True the APK file on the host will be prefered if it is a valid version and ABI, if not it will fall back to the version on the targer. When set to False the target version is prefered. default: ``True`` force_install : boolean Always re-install the APK, even if matching version is found already installed on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When this is set, check_apk is ignored. uninstall_apk : boolean If ``True``, will uninstall workload's APK as part of teardown. exact_abi : boolean If ``True``, workload will check that the APK matches the target device ABI, otherwise any APK found will be used. clear_data_on_reset : boolean If set to ``False``, this will prevent WA from clearing package data for this workload prior to running it. default: ``True`` markers_enabled : boolean If ``True``, UX_PERF action markers will be emitted to logcat during the test run. clean_assets : boolean If ``True`` pushed assets will be deleted at the end of each iteration force_push_assets : boolean If ``True`` always push assets on each iteration, even if the assets already exists in the device path video_source : str Determines where to play the video from. This can either be from the YouTube home, my videos section, trending videos or found in search. allowed values: ``'home'``, ``'my_videos'``, ``'search'``, ``'trending'`` default: ``'search'`` search_term : str The search term to use when ``video_source`` is set to ``search``. Ignored otherwise. default: ``'Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film'``