Instruments

apk_version
cpufreq
delay
dmesg
energy_measurement
execution_time
file_poller
fps
hwmon
interrupts
perf
perfetto
proc_stat
screen_capture
serialmon
sysfs_extractor
trace-cmd

apk_version 

Extracts APK versions for workloads that have them.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

cpufreq 

Collects dynamic frequency (DVFS) settings before and after workload execution.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

paths:

type: 'list_of_strs'

A list of paths to be pulled from the device. These could be directories as well as files.

global alias: 'sysfs_extract_dirs'

use_tmpfs:

type: 'boolean'

Specifies whether tmpfs should be used to cache sysfile trees and then pull them down as a tarball. This is significantly faster then just copying the directory trees from the device directly, but requires root and may not work on all devices. Defaults to True if the device is rooted and False if it is not.

tmpfs_mount_point:

type: 'str'

Mount point for tmpfs partition used to store snapshots of paths.

tmpfs_size:

type: 'str'

Size of the tempfs partition.

default: '32m'

delay 

This instrument introduces a delay before beginning a new spec, a new job or before the main execution of a workload.

The delay may be specified as either a fixed period or a temperature threshold that must be reached.

Optionally, if an active cooling solution is available on the device to speed up temperature drop between runs, it may be controlled using this instrument.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

temperature_file:

type: 'str'

Full path to the sysfile on the target that contains the target’s temperature.

global alias: 'thermal_temp_file'

default: '/sys/devices/virtual/thermal/thermal_zone0/temp'

temperature_timeout:

type: 'integer'

The timeout after which the instrument will stop waiting even if the specified threshold temperature is not reached. If this timeout is hit, then a warning will be logged stating the actual temperature at which the timeout has ended.

global alias: 'thermal_timeout'

default: 600

temperature_poll_period:

type: 'integer'

How long to sleep (in seconds) between polling current target temperature.

global alias: 'thermal_sleep_time'

default: 5

temperature_between_specs:

type: 'integer'

Temperature (in target-specific units) the target must cool down to before the iteration spec will be run.

If this is set to 0 then the devices initial temperature will used as the threshold.

Note

This cannot be specified at the same time as fixed_between_specs

global alias: 'thermal_threshold_between_specs'

fixed_between_specs:

type: 'integer'

How long to sleep (in seconds) before starting a new workload spec.

Note

This cannot be specified at the same time as temperature_between_specs

global alias: 'fixed_delay_between_specs'

temperature_between_jobs:

type: 'integer'

Temperature (in target-specific units) the target must cool down to before the next job will be run.

If this is set to 0 then the devices initial temperature will used as the threshold.

Note

This cannot be specified at the same time as fixed_between_jobs

aliases: 'temperature_between_iterations'

global alias: 'thermal_threshold_between_jobs'

fixed_between_jobs:

type: 'integer'

How long to sleep (in seconds) before starting each new job.

Note

This cannot be specified at the same time as temperature_between_jobs

aliases: 'fixed_between_iterations'

global alias: 'fixed_delay_between_jobs'

fixed_before_start:

type: 'integer'

How long to sleep (in seconds) after setup for an iteration has been performed but before running the workload.

Note

This cannot be specified at the same time as temperature_before_start

global alias: 'fixed_delay_before_start'

temperature_before_start:

type: 'integer'

Temperature (in device-specific units) the device must cool down to just before the actual workload execution (after setup has been performed).

Note

This cannot be specified at the same time as fixed_between_jobs

global alias: 'thermal_threshold_before_start'

active_cooling:

type: 'boolean'

This instrument supports an active cooling solution while waiting for the device temperature to drop to the threshold. If you wish to use this feature please ensure the relevant module is installed on the device.

default: False

dmesg 

Collected dmesg output before and during the run.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

loglevel:

type: 'integer'

Set loglevel for console output.

allowed values: 0, 1, 2, 3, 4, 5, 6, 7

energy_measurement 

This instrument is designed to be used as an interface to the various energy measurement instruments located in devlib.

This instrument should be used to provide configuration for any of the Energy Instrument Backends rather than specifying configuration directly.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

instrument: (mandatory)

type: 'str'

Specify the energy instruments to be enabled.

allowed values: 'daq', 'energy_probe', 'acme_cape', 'monsoon', 'juno_readenergy', 'arm_energy_probe'

instrument_parameters:

type: 'OrderedDict'

Specify the parameters used to initialize the desired instruments. To see parameters available for a particular instrument, run

wa show <instrument name>

See help for instrument parameter to see available options for <instrument name>.

default:

{

}

sites:

type: 'list_or_string'

Specify which sites measurements should be collected from, if not specified the measurements will be collected for all available sites.

kinds:

type: 'list_or_string'

Specify the kinds of measurements should be collected, if not specified measurements will be collected for all available kinds.

channels:

type: 'list_or_string'

Specify the channels to be collected, if not specified the measurements will be collected for all available channels.

execution_time 

Measure how long it took to execute the run() methods of a Workload.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

file_poller 

Polls the given files at a set sample interval. The values are output in CSV format.

This instrument places a file called poller.csv in each iterations result directory. This file will contain a timestamp column which will be in uS, the rest of the columns will be the contents of the polled files at that time.

This instrument will strip any commas or new lines for the files’ values before writing them.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

sample_interval:

type: 'integer'

The interval between samples in mS.

default: 1000

files: (mandatory)

type: 'list_or_string'

A list of paths to the files to be polled

labels:

type: 'list_or_string'

A list of lables to be used in the CSV output for the corresponding files. This cannot be used if a * wildcard is used in a path.

align_with_ftrace:

type: 'boolean'

Insert a marker into ftrace that aligns with the first timestamp. During output processing, extract the marker and use it’s timestamp to adjust the timestamps in the collected csv so that they align with ftrace.

default: False

as_root:

type: 'boolean'

Whether or not the poller will be run as root. This should be used when the file you need to poll can only be accessed by root.

default: False

fps 

Measures Frames Per Second (FPS) and associated metrics for a workload.

Note

This instrument depends on pandas Python library (which is not part of standard WA dependencies), so you will need to install that first, before you can use it.

Android L and below use SurfaceFlinger to calculate the FPS data. Android M and above use gfxinfo to calculate the FPS data.

SurfaceFlinger: The view is specified by the workload as view attribute. This defaults to 'SurfaceView' for game workloads, and None for non-game workloads (as for them FPS mesurement usually doesn’t make sense). Individual workloads may override this.

gfxinfo: The view is specified by the workload as package attribute. This is because gfxinfo already processes for all views in a package.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

drop_threshold:

type: 'numeric'

Data points below this FPS will be dropped as they do not constitute “real” gameplay. The assumption being that while actually running, the FPS in the game will not drop below X frames per second, except on loading screens, menus, etc, which should not contribute to FPS calculation.

default: 5

keep_raw:

type: 'boolean'

If set to True, this will keep the raw dumpsys output in the results directory (this is maily used for debugging) Note: frames.csv with collected frames data will always be generated regardless of this setting.

default: False

crash_threshold:

type: 'float'

Specifies the threshold used to decided whether a measured/expected frames ration indicates a content crash. E.g. a value of 0.75 means the number of actual frames counted is a quarter lower than expected, it will treated as a content crash.

If set to zero, no crash check will be performed.

default: 0.7

period:

type: 'float'

Specifies the time period between polling frame data in seconds when collecting frame data. Using a lower value improves the granularity of timings when recording actions that take a short time to complete. Note, this will produce duplicate frame data in the raw dumpsys output, however, this is filtered out in frames.csv. It may also affect the overall load on the system.

The default value of 2 seconds corresponds with the NUM_FRAME_RECORDS in android/services/surfaceflinger/FrameTracker.h (as of the time of writing currently 128) and a frame rate of 60 fps that is applicable to most devices.

constraint: value > 0

default: 2

force_surfaceflinger:

type: 'boolean'

By default, the method to capture fps data is based on Android version. If this is set to true, force the instrument to use the SurfaceFlinger method regardless of its Android version.

default: False

hwmon 

Hardware Monitor (hwmon) is a generic Linux kernel subsystem, providing access to hardware monitoring components like temperature or voltage/current sensors.

Data from hwmon that are a snapshot of a fluctuating value, such as temperature and voltage, are reported once at the beginning and once at the end of the workload run. Data that are a cumulative total of a quantity, such as energy (which is the cumulative total of power consumption), are reported as the difference between the values at the beginning and at the end of the workload run.

There is currently no functionality to filter sensors: all of the available hwmon data will be reported.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

interrupts 

Pulls the /proc/interrupts file before and after workload execution and diffs them to show what interrupts occurred during that time.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

perf 

Perf is a Linux profiling with performance counters. Simpleperf is an Android profiling tool with performance counters.

It is highly recomended to use perf_type = simpleperf when using this instrument on android devices since it recognises android symbols in record mode and is much more stable when reporting record .data files. For more information see simpleperf documentation at: https://android.googlesource.com/platform/system/extras/+/master/simpleperf/doc/README.md

Performance counters are CPU hardware registers that count hardware events such as instructions executed, cache-misses suffered, or branches mispredicted. They form a basis for profiling applications to trace dynamic control flow and identify hotspots.

perf accepts options and events. If no option is given the default ‘-a’ is used. For events, the default events for perf are migrations and cs. The default events for simpleperf are raw-cpu-cycles, raw-l1-dcache, raw-l1-dcache-refill, raw-instructions-retired. They both can be specified in the config file.

Events must be provided as a list that contains them and they will look like this

(for perf_type = perf ) perf_events = ['migrations', 'cs']
(for perf_type = simpleperf) perf_events = ['raw-cpu-cycles', 'raw-l1-dcache']

Events can be obtained by typing the following in the command line on the device

perf list
simpleperf list

Whereas options, they can be provided as a single string as following

perf_options = '-a -i'
perf_options = '--app com.adobe.reader'

Options can be obtained by running the following in the command line

man perf-stat

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

perf_type:

type: 'str'

Specifies which type of perf binaries to install. Use simpleperf for collecting perf data on android systems.

global alias: 'perf_type'

allowed values: 'perf', 'simpleperf'

default: 'perf'

command:

type: 'str'

Specifies which perf command to use. If in record mode report command will also be executed and results pulled from target along with raw data file

global alias: 'perf_command'

allowed values: 'stat', 'record'

default: 'stat'

events:

type: 'list_of_strs'

Specifies the events to be counted.

global alias: 'perf_events'

optionstring:

type: 'list_or_string'

Specifies options to be used for the perf command. This may be a list of option strings, in which case, multiple instances of perf will be kicked off – one for each option string. This may be used to e.g. collected different events from different big.LITTLE clusters. In order to profile a particular application process for android with simpleperf use the –app option e.g. –app com.adobe.reader

global alias: 'perf_options'

default: '-a'

report_option_string:

type: 'str'

Specifies options to be used to gather report when record command is used. It’s highly recommended to use perf_type simpleperf when running on android devices as reporting options are unstable with perf

global alias: 'perf_report_options'

run_report_sample:

type: 'boolean'

If true, run ‘perf/simpleperf report-sample’. It only works with the record command.

default: False

report_sample_options:

type: 'str'

Specifies options to pass to report-samples when run_report_sample is true.

labels:

type: 'list_of_strs'

Provides labels for perf/simpleperf output for each optionstring. If specified, the number of labels must match the number of optionstrings.

global alias: 'perf_labels'

force_install:

type: 'boolean'

always install perf binary even if perf is already present on the device.

default: False

perfetto 

perfetto is an instrument that interacts with Google’s Perfetto tracing infrastructure.

From Perfetto’s website: Perfetto is a production-grade open-source stack for performance instrumentation and trace analysis. It offers services and libraries for recording system-level and app-level traces, native + java heap profiling, a library for analyzing traces using SQL and a web-based UI to visualize and explore multi-GB traces.

The instrument either requires Perfetto to be present on the target device or the standalone tracebox binary to be built from source and included in devlib’s Package Bin directory. For more information, consult the PerfettoCollector documentation in devlib.

More information can be found on https://perfetto.dev/

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

config: (mandatory)

type: 'str'

Path to the Perfetto trace config file.

All the Perfetto-specific tracing configuration should be done inside that file. This config option should just take a full filesystem path to where the config can be found.

force_tracebox:

type: 'boolean'

Install tracebox even if traced is already running on the target device. If set to true, the tracebox binary needs to be placed in devlib’s Package Bin directory.

default: False

proc_stat 

Collect CPU load information from /proc/stat.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

period:

type: 'integer'

Time (in seconds) between collections.

constraint: value > 0

default: 5

screen_capture 

A simple instrument which captures the screen on the target devices with a user-specified period.

Please note that if a too short period is specified, then this instrument will capture the screen as fast as possible, rather than at the specified periodicity.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

period:

type: 'integer'

Period (in seconds) at which to capture the screen on the target.

default: 10

serialmon 

Records the traffic on a serial connection

The traffic on a serial connection is monitored and logged to a file. In the event that the device is reset, the instrument will stop monitoring during the reset, and will reconnect once the reset has completed. This is to account for devices (i.e., the Juno) which utilise the serial connection to reset the board.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

serial_port:

type: 'str'

The serial device to monitor.

default: '/dev/ttyS0'

baudrate:

type: 'integer'

The baud-rate to use when connecting to the serial connection.

default: 115200

sysfs_extractor 

Collects the contest of a set of directories, before and after workload execution and diffs the result.

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

paths: (mandatory)

type: 'list_of_strs'

A list of paths to be pulled from the device. These could be directories as well as files.

global alias: 'sysfs_extract_dirs'

use_tmpfs:

type: 'boolean'

Specifies whether tmpfs should be used to cache sysfile trees and then pull them down as a tarball. This is significantly faster then just copying the directory trees from the device directly, but requires root and may not work on all devices. Defaults to True if the device is rooted and False if it is not.

tmpfs_mount_point:

type: 'str'

Mount point for tmpfs partition used to store snapshots of paths.

tmpfs_size:

type: 'str'

Size of the tempfs partition.

default: '32m'

trace-cmd 

trace-cmd is an instrument which interacts with ftrace Linux kernel internal tracer

From trace-cmd man page:

trace-cmd command interacts with the ftrace tracer that is built inside the Linux kernel. It interfaces with the ftrace specific files found in the debugfs file system under the tracing directory.

trace-cmd reads a list of events it will trace, which can be specified in the config file as follows

trace_events = ['irq*', 'power*']

If no event is specified, a default set of events that are generally considered useful for debugging/profiling purposes will be enabled.

The list of available events can be obtained by rooting and running the following command line on the device

trace-cmd list

You may also specify trace_buffer_size setting which must be an integer that will be used to set the ftrace buffer size. It will be interpreted as KB:

trace_cmd_buffer_size = 8000

The maximum buffer size varies from device to device, but there is a maximum and trying to set buffer size beyond that will fail. If you plan on collecting a lot of trace over long periods of time, the buffer size will not be enough and you will only get trace for the last portion of your run. To deal with this you can set the trace_mode setting to 'record' (the default is 'start'):

trace_cmd_mode = 'record'

This will cause trace-cmd to trace into file(s) on disk, rather than the buffer, and so the limit for the max size of the trace is set by the storage available on device. Bear in mind that 'record' mode is more intrusive than the default, so if you do not plan on generating a lot of trace, it is best to use the default 'start' mode.

Note

Mode names correspond to the underlying trace-cmd executable’s command used to implement them. You can find out more about what is happening in each case from trace-cmd documentation: https://lwn.net/Articles/341902/.

This instrument comes with an trace-cmd binary that will be copied and used on the device, however post-processing will be, by default, done on-host and you must have trace-cmd installed and in your path. On Ubuntu systems, this may be done with:

sudo apt-get install trace-cmd

Alternatively, you may set report_on_target parameter to True to enable on-target processing (this is useful when running on non-Linux hosts, but is likely to take longer and may fail on particularly resource-constrained targets).

parameters

cleanup_assets:

type: 'boolean'

If True, assets that are deployed or created by the plugin will be removed again from the device.

aliases: 'clean_up'

global alias: 'cleanup_assets'

default: True

events:

type: 'list_of_strs'

Specifies the list of events to be traced. Each event in the list will be passed to trace-cmd with -e parameter and must be in the format accepted by trace-cmd.

global alias: 'trace_events'

default: ['sched*', 'irq*', 'power*', 'thermal*']

functions:

type: 'list_of_strs'

Specifies the list of functions to be traced.

global alias: 'trace_functions'

buffer_size:

type: 'integer'

Attempt to set ftrace buffer size to the specified value (in KB). Default buffer size may need to be increased for long-running workloads, or if a large number of events have been enabled. Note: there is a maximum size that the buffer can be set, and that varies from device to device. Attempting to set buffer size higher than this will fail. In that case, this instrument will set the size to the highest possible value by going down from the specified size in buffer_size_step intervals.

global alias: 'trace_buffer_size'

top_buffer_size:

type: 'integer'

The same as buffer_size except it sets the size of the top-level buffer instead of the devlib one. If left unset, it will default to the same as the devlib buffer size.

global alias: 'trace_top_buffer_size'

buffer_size_step:

type: 'integer'

Defines the decremental step used if the specified buffer_size could not be set. This will be subtracted form the buffer size until set succeeds or size is reduced to 1MB.

global alias: 'trace_buffer_size_step'

default: 1000

report:

type: 'boolean'

Specifies whether reporting should be performed once the binary trace has been generated.

default: True

no_install:

type: 'boolean'

Do not install the bundled trace-cmd and use the one on the device instead. If there is not already a trace-cmd on the device, an error is raised.

default: False

report_on_target:

type: 'boolean'

When enabled generation of reports will be done host-side because the generated file is very large. If trace-cmd is not available on the host device this setting can be disabled and the report will be generated on the target device.

Note

This requires the latest version of trace-cmd to be installed on the host (the one in your distribution’s repos may be too old).

default: False