================================= What's New in Workload Automation ================================= ------------- Version 2.6.0 ------------- .. note:: Users who are currently using the GitHub master version of WA should uninstall the existing version before upgrading to avoid potential issues. Additions: ########## Workloads ~~~~~~~~~ - ``AdobeReader``: A workload that carries out following typical productivity tasks. These include opening a file, performing various gestures and zooms on screen and searching for a predefined set of strings. - ``octaned8``: A workload to run the binary (non-browser) version of the JS benchmark Octane. - ``GooglePlayBooks``: A workload to perform standard productivity tasks with Google Play Books. This workload performs various tasks, such as searching for a book title online, browsing through a book, adding and removing notes, word searching, and querying information about the book. - ``GooglePhotos``: A workload to perform standard productivity tasks with Google Photos. Carries out various tasks, such as browsing images, performing zooms, and post-processing the image. - ``GoogleSlides``: Carries out various tasks, such as creating a new presentation, adding text, images, and shapes, as well as basic editing and playing a slideshow. - ``Youtube``: The workload plays a video, determined by the ``video_source`` parameter. While the video is playing, some common actions such as video seeking, pausing playback and navigating the comments section are performed. - ``Skype``: Replacement for the ``skypevideo`` workload. Logs into Skype and initiates a voice or video call with a contact. Framework ~~~~~~~~~ - ``AndroidUxPerfWorkload``: Added a new workload class to encapsulate functionality common to all uxperf workloads. - ``UxPerfUiAutomation``: Added class which contains methods specific to UX performance testing. - ``get-assets``: Added new script and command to retrieve external assets for workloads Results Processors ~~~~~~~~~~~~~~~~~~~ - ``uxperf``: Parses device logcat for `UX_PERF` markers to produce performance metrics for workload actions using specified instrumentation. Other ~~~~~ - ``State Detection``: Added feature to use visual state detection to verify the state of a workload after setup and run. Fixes/Improvements: ################### Documentation ~~~~~~~~~~~~~~ - ``Revent``: Added file structure to the documentation. - Clarified documentation regarding binary dependencies. - Updated documentation with ``create`` and ``get-assets`` commands. Instruments ~~~~~~~~~~~~ - ``sysfs_extractor``: Fixed error when `tar.gz` file already existed on device, now overwrites. - ``cpufreq``: Fixed error when `tar.gz` file already existed on device, now overwrites. - ``file-poller``: - Improved csv output. - Added error checking and reporting. - Changed ``files`` to be a mandatory parameter. - ``fps``: - Added a new parameter to fps instrument to specify the time period between calls to ``dumpsys SurfaceFlinger --latency`` when collecting frame data. - Added gfxinfo methods to obtain fps stats. Auto detects and uses appropriate method via android version of device. - Fixed issue with regex. - Now handles empty frames correctly. - ``energy_model``: Ensures that the ``ui`` runtime parameter is only set for ChromeOS devices. - ``ftrace``: Added support to handle traces collected by both WA and devlib. - ``Perf``: Updated 32bit binary file for little endian devices. Resource Getters ~~~~~~~~~~~~~~~~ - ``http_getter``: Now used to try and find executables files from a provided ``remove_assets_url``. Result Processors ~~~~~~~~~~~~~~~~~ - ``cpu_states``: Fixes using stand-alone script with timeline option. Workloads ~~~~~~~~~ - ``antutu``: Fixed setting permissions of ``FINE_LOCATION`` on some devices. - ``bbench`` Fixed handling of missing results. - ``camerarecord``: - Added frame stats collection through dumpsys gfxinfo. - Added possibility to select slow_motion recording mode. - ``Geekbench``: - Fixed output file listing causing pull failure. - Added support for Geekbench 4. - ``recentfling``: - Fixed issue when binaries were not uninstalled correctly. - Scripts are now deployed via ``install()`` to ensure they are executable. - Fixed handling of when a PID file is deleted before reaching processing results stage. - Added parameter to not start any apps before flinging. - ``rt-app``: Added camera recorder simulation. - ``sysbench``: Added arm64 binary. - ``Vellamo``: Fixed capitalization in part of UIAutomation to prevent potential issues. - ``Spec2000``: Now uses WA deployed version of busybox. - ``NetStat``: Updated to support new default logcat format in Android 6. - ``Dex2oat``: Now uses root if available. Framework ~~~~~~~~~ - ``adb_shell``: - Fixed issue when using single quoted command with ``adb_shell``. - Correctly forward stderror to the caller for newer version of adb. - ``revent`` - Added ``-S`` argument to "record" command to automatically record a screen capture after a recording is completed. - Fixed issue with multiple iterations of a revent workload. - Added ``-s`` option to executable to allow waiting on stdin. - Removed timeout in command as ``-s`` is specified. - Revent recordings can now be parsed and used within WA. - Fixed issue when some recordings wouldn't be retrieved correctly. - Timeout is now based on recording duration. - Added `magic` and file version to revent files. Revent files should now start with ``REVENT`` followed by the file format version. - Added support for gamepad recording. This type of recording contains only the events from a gamepad device (which is automatically identified). - A ``mode`` field has been added to the recording format to help distinguish between the normal and gamepad recording types. - Added ``-g`` option to ``record`` command to expose the gamepad recording mode. - The structure of revent code has undergone a major overhaul to improve maintainability and robustness. - More detailed ``info`` command output. - Updated Makefile to support debug/production builds. - ``Android API``: Upgraded Android API level from 17 to 18. - ``uiautomator``: The window hierarchy is now dumped to a file when WA fails on android devices. - ``AndroidDevice``: - Added support for downgrading when installing an APK. - Added a ``broadcast_media_mounted`` method to force a re-index of the mediaserver cache for a specified directory. - Now correctly handles ``None`` output for ``get_pids_of()`` when there are no running processes with the specified name. - Renamed the capture method from ``capture_view_hierachy`` to ``capture_ui_hierarchy``. - Changed the file extension of the capture file to ``.uix`` - Added ``-rf`` to delete_files to be consistent with ``LinuxDevice``. - ``LinuxDevice``: Now ensures output from both stdout and etderr is propagated in the event of a DeviceError. - ``APKWorkload``: - Now ensure APKs are replaced properly when reinstalling. - Now checks APK version and ABI when installing. - Fixed error on some devices when trying to grant permissions that were already granted. - Fixed some permissions not being granted. - Now allows disabling the main activity launch in setup (required for some apps). - Added parameter to clear data on reset (default behaviour unchanged). - Ignores exception for non-fatal permission grant failure. - Fixed issue of multiple versions of the same workload failing to find their APK. - Added method to ensure a valid apk version is used within a workload. - Updated how APK resolution is performed to maximise likelihood of a workload running. - When ``check_apk`` is ``True`` will prefer host APK and if no suitable APK is found, will use target APK if the correct version is present. When ``False`` will prefer target apk if it is a valid version otherwise will fallback to host APK. - ``RunConfiguration``: Fixed disabling of instruments in workload specs. - ``Devices``: - Added network connectivity check for devices. - Subclasses can now set ``requires_network`` to ``True`` and network connectivity check will be performed during ``setup()``. - ``Workloads``: - Added network check methods. - Fixed versions to be backwards compatible. - Updated workload versions to match APK files. - Fixed issues with calling super. - ``Assets``: Added script to retrieve external assets for workloads. - ``Execution``: Added a ``clean_up`` global config option to delete WA files from devices. - ``Runner``: No longer takes a screenshot or dump of UI hierarchy for some errors when unnecessary, e.g. host errors. - ``core``: Constraints and allowed values are now checked when set instead of when validating. - ``FpsProcessor``: - Added requirement on ``filtered_vsyncs_to_compose`` for ``total_vsync metric``. - Removed misleading comment in class description. - ``BaseUiAutomation``: Added new Marker API so workloads generate start and end markers with a string name. - ``AndroidUiAutoBenchmark``: Automatically checks for known package versions that don't work well with AndroidUiAutoBenchmark workloads. Other ~~~~~ - Updated setup.py url to be a valid URI. - Fixed workload name in big.Little sample agenda. Incompatible changes #################### Framework ~~~~~~~~~ - ``check_abi``: Now renamed to ``exact_abi``, is used to ensure that if enabled, only an apk containing no native code or code designed for the devices primary abi is use. - ``AndroidDevice``: Renamed ``supported_eabis`` property to ``supported_abis`` to be consistent with linux devices. Workloads ~~~~~~~~~~ - ``skypevideo``: Workload removed and replaced with ``skype`` workload. ------------- Version 2.5.0 ------------- Additions: ########## Instruments ~~~~~~~~~~~ - ``servo_power``: Added support for chromebook servo boards. - ``file_poller``: polls files and outputs a CSV of their values over time. - ``systrace``: The Systrace tool helps analyze the performance of your application by capturing and displaying execution times of your applications processes and other Android system processes. Workloads ~~~~~~~~~ - ``blogbench``: Blogbench is a portable filesystem benchmark that tries to reproduce the load of a real-world busy file server. - ``stress-ng``: Designed to exercise various physical subsystems of a computer as well as the various operating system kernel interfaces. - ``hwuitest``: Uses hwuitest from AOSP to test rendering latency on Android devices. - ``recentfling``: Tests UI jank on android devices. - ``apklaunch``: installs and runs an arbitrary apk file. - ``googlemap``: Launches Google Maps and replays previously recorded interactions. Framework ~~~~~~~~~ - ``wlauto.utils.misc``: Added ``memoised`` function decorator that allows caching of previous function/method call results. - Added new ``Device`` APIs: - ``lsmod``: lists kernel modules - ``insmod``: inserts a kernel module from a ``.ko`` file on the host. - ``get_binary_path``: Checks ``binary_directory`` for the wanted binary, if it is not found there it will try to use ``which`` - ``install_if_needed``: Will only install a binary if it is not already on the target. - ``get_device_model``: Gets the model of the device. - ``wlauto.core.execution.ExecutionContext``: - ``add_classfiers``: Allows adding a classfier to all metrics for the current result. Other ~~~~~ - Commands: - ``record``: Simplifies recording revent files. - ``replay``: Plays back revent files. Fixes/Improvements: ################### Devices ~~~~~~~ - ``juno``: - Fixed ``bootargs`` parameter not being passed to ``_boot_via_uboot``. - Removed default ``bootargs`` - ``gem5_linux``: - Added ``login_prompt`` and ``login_password_prompt`` parameters. - ``generic_linux``: ABI is now read from the target device. Instruments ~~~~~~~~~~~ - ``trace-cmd``: - Added the ability to report the binary trace on the target device, removing the need for ``trace-cmd`` binary to be present on the host. - Updated to handle messages that the trace for a CPU is empty. - Made timeout for pulling trace 1 minute at minimum. - ``perf``: per-cpu statistics now get added as metrics to the results (with a classifier used to identify the cpu). - ``daq``: - Fixed bug where an exception would be raised if ``merge_channels=False`` - No longer allows duplicate channel labels - ``juno_energy``: - Summary metrics are now calculated from the contents of ``energy.csv`` and added to the overall results. - Added a ``strict`` parameter. When this is set to ``False`` the device check during validation is omitted. - ``sysfs_extractor``: tar and gzip are now performed separately to solve permission issues. - ``fps``: - Now only checks for crashed content if ``crash_check`` is ``True``. - Can now process multiple ``view`` attributes. - ``hwmon``: Sensor naming fixed, they are also now added as result classifiers Resource Getters ~~~~~~~~~~~~~~~~ - ``extension_asset``: Now picks up the path to the mounted filer from the ``remote_assets_path`` global setting. Result Processors ~~~~~~~~~~~~~~~~~ - ``cpustates``: - Added the ability to configure how a missing ``START`` marker in the trace is handled. - Now raises a warning when there is a ``START`` marker in the trace but no ``STOP`` marker. - Exceptions in PowerStateProcessor no longer stop the processing of the rest of the trace. - Now ensures a known initial state by nudging each CPU to bring it out of idle and writing starting CPU frequencies to the trace. - Added the ability to create a CPU utilisation timeline. - Fixed issues with getting frequencies of hotplugged CPUs - ``csv``: Zero-value classifieres are no longer converted to an empty entry. - ``ipynb_exporter``: Default template no longer shows a blank plot for workloads without ``summary_metrics`` Workloads ~~~~~~~~~ - ``vellamo``: - Added support for v3.2.4. - Fixed getting values from logcat. - ``cameracapture``: Updated to work with Android M+. - ``camerarecord``: Updated to work with Android M+. - ``lmbench``: - Added the output file as an artifact. - Added taskset support - ``antutu`` - Added support for v6.0.1 - ``ebizzy``: Fixed use of ``os.path`` to ``self.device.path``. - ``bbench``: Fixed browser crashes & permissions issues on android M+. - ``geekbench``: - Added check whether device is rooted. - ``manual``: Now only uses logcat on Android devices. - ``applaunch``: - Fixed ``cleanup`` not getting forwarded to script. - Added the ability to stress IO during app launch. - ``dhrystone``: Now uses WA's resource resolution to find it's binary so it uses the correct ABI. - ``glbench``: Updated for new logcat formatting. Framework ~~~~~~~~~ - ``ReventWorkload``: - Now kills all revent instances on teardown. - Device model name is now used when searching for revent files, falling back to WA device name. - ``BaseLinuxDevice``: - ``killall`` will now run as root by default if the device is rooted. - ``list_file_systems`` now handles blank lines. - All binaries are now installed into ``binaries_directory`` this allows.. - Busybox is now deployed on non-root devices. - gzipped property files are no zcat'ed - ``LinuxDevice``: - ``kick_off`` no longer requires root. - ``kick_off`` will now run as root by default if the device is rooted. - No longer raises an exception if a connection was dropped during a reboot. - Added a delay before polling for a connection to avoid re-connecting to a device that is still in the process of rebooting. - ``wlauto.utils.types``: ``list_or_string`` now ensures that elements of a list are strings. - ``AndroidDevice``: - ``kick_off`` no longer requires root. - Build props are now gathered via ``getprop`` rather than trying to parse build.prop directly. - WA now pushes its own ``sqlite3`` binary. - Now uses ``content`` instead of ``settings`` to get ``ANDROID_ID`` - ``swipe_to_unlock`` parameter is now actually used. It has been changed to take a direction to accomodate various devices. - ``ensure_screen_is_on`` will now also unlock the screen if swipe_to_unlock is set. - Fixed use of variables in as_root=True commands. - ``get_pids_of`` now used ``busybox grep`` since as of Android M+ ps cannot filter by process name anymore. - Fixed installing APK files with whitespace in their path/name. - ``adb_shell``: - Fixed handling of line breaks at the end of command output. - Newline separator is now detected from the target. - As of ADB v1.0.35, ADB returns the return code of the command run. WA now handles this correctly. - ``ApkWorkload``: - Now attempts to grant all runtime permissions for devices on Android M+. - Can now launch packages that don't have a launch activity defined. - Package version is now added to results as a classifier. - Now clears app data if an uninstall failed to ensure it starts from a known state. - ``wlauto.utils.ipython``: Updated to work with ipython v5. - ``Gem5Device``: - Added support for deploying the ``m5`` binary. - No longer waits for the boot animation to finish if it has been disabled. - Fixed runtime error caused by lack of kwargs. - No longer depends on ``busybox``. - Split out commands to resize shell to ``resize_shell``. - Now tries to connect to the shell up to 10 times. - No longer renames gzipped files. - Agendas: - Now errors when an agenda key is empty. - ``wlauto.core.execution.RunInfo``: ``run_name`` will now default to ``{output_folder}_{date}_{time}``. - Extensions: - Two different parameters can now have the same global alias as long as they their types match. - You can no longer ``override`` parameters that are defined at the same level. - ``wlauto.core.entry_point``: Now gives a better error when a config file doesn't exist. - ``wlauto.utils.misc``: Added ``aarch64`` to list for arm64 ABI. - ``wlauto.core.resolver``: Now shows what version was being search for when a resource is not found. - Will no longer start instruments ect. if a run has no workload specs. - ``wlauto.utils.uboot``: Now detects uboot version to use correct line endings. - ``wlauto.utils.trace_cmd``: Added a parser for sched_switch events. Other ~~~~~ - Updated to pylint v1.5.1 - Rebuilt ``busybox`` binaries to prefer built-in applets over system binaries. - ``BaseUiAutomation``: Added functions for checking version strings. Incompatible changes #################### Instruments ~~~~~~~~~~~ - ``apk_version``: Removed, use result classifiers instead. Framework ~~~~~~~~~ - ``BaseLinuxDevice``: Removed ``is_installed`` use ``install_if_needed`` and ``get_binary_path`` instead. - ``LinuxDevice``: Removed ``has_root`` method, use ``is_rooted`` instead. - ``AndroidDevice``: ``swipe_to_unlock`` method replaced with ``perform_unlock_swipe``. ------------- Version 2.4.0 ------------- Additions: ########## Devices ~~~~~~~~ - ``gem5_linux`` and ``gem5_android``: Interfaces for Gem5 simulation environment running Linux and Android respectively. - ``XE503C1211``: Interface for Samsung XE503C12 Chromebooks. - ``chromeos_test_image``: Chrome OS test image device. An off the shelf device will not work with this device interface. Instruments ~~~~~~~~~~~~ - ``freq_sweep``: Allows "sweeping" workloads across multiple CPU frequencies. - ``screenon``: Ensures screen is on, before each iteration, or periodically on Android devices. - ``energy_model``: This instrument can be used to generate an energy model for a device based on collected power and performance measurments. - ``netstats``: Allows monitoring data sent/received by applications on an Android device. Modules ~~~~~~~ - ``cgroups``: Allows query and manipulation of cgroups controllers on a Linux device. Currently, only cpusets controller is implemented. - ``cpuidle``: Implements cpuidle state discovery, query and manipulation for a Linux device. This replaces the more primitive get_cpuidle_states method of LinuxDevice. - ``cpufreq`` has now been split out into a device module Reasource Getters ~~~~~~~~~~~~~~~~~ - ``http_assets``: Downloads resources from a web server. Results Processors ~~~~~~~~~~~~~~~~~~~ - ``ipynb_exporter``: Generates an IPython notebook from a template with the results and runs it. - ``notify``: Displays a desktop notification when a run finishes (Linux only). - ``cpustates``: Processes power ftrace to produce CPU state and parallelism stats. There is also a script to invoke this outside of WA. Workloads ~~~~~~~~~ - ``telemetry``: Executes Google's Telemetery benchmarking framework - ``hackbench``: Hackbench runs tests on the Linux scheduler - ``ebizzy``: This workload resembles common web server application workloads. - ``power_loadtest``: Continuously cycles through a set of browser-based activities and monitors battery drain on a device (part of ChromeOS autotest suite). - ``rt-app``: Simulates configurable real-time periodic load. - ``linpack-cli``: Command line version of linpack benchmark. - ``lmbench``: A suite of portable ANSI/C microbenchmarks for UNIX/POSIX. - ``stream``: Measures memory bandwidth. - ``iozone``: Runs a series of disk I/O performance tests. - ``androbench``: Measures the storage performance of device. - ``autotest``: Executes tests from ChromeOS autotest suite. Framework ~~~~~~~~~ - ``wlauto.utils``: - Added ``trace_cmd``, a generic trace-cmd paraser. - Added ``UbootMenu``, allows navigating Das U-boot menu over serial. - ``wlauto.utils.types``: - ``caseless_string``: Behaves exactly like a string, except this ignores case in comparisons. It does, however, preserve case. - ``list_of``: allows dynamic generation of type-safe list types based on an existing type. - ``arguments``: represents arguments that are passed on a command line to an application. - ``list-or``: allows dynamic generation of types that accept either a base type or a list of base type. Using this ``list_or_integer``, ``list_or_number`` and ``list_or_bool`` were also added. - ``wlauto.core.configuration.WorkloadRunSpec``: - ``copy``: Allows making duplicates of ``WorkloadRunSpec``'s - ``wlatuo.utils.misc``: - ``list_to_ranges`` and ``ranges_to_list``: convert between lists of integers and corresponding range strings, e.g. between [0,1,2,4] and '0-2,4' - ``list_to_mask`` and ``mask_to_list``: convert between lists of integers and corresponding integer masks, e.g. between [0,1,2,4] and 0x17 - ``wlauto.instrumentation``: - ``instrument_is_enabled``: Returns whether or not an instrument is enabled for the current job. - ``wlauto.core.result``: - Added "classifiers" field to Metric objects. This is a dict mapping classifier names (arbitrary strings) to corresponding values for that specific metrics. This is to allow extensions to add extension-specific annotations to metric that could be handled in a generic way (e.g. by result processors). They can also be set in agendas. - Failed jobs will now be automatically retired - Implemented dynamic device modules that may be loaded automatically on device initialization if the device supports them. - Added support for YAML configs. - Added ``initialze`` and ``finalize`` methods to workloads. - ``wlauto.core.ExecutionContext``: - Added ``job_status`` property that returns the status of the currently running job. Fixes/Improvements ################## Devices ~~~~~~~~ - ``tc2``: Workaround for buffer overrun when loading large initrd blob. - ``juno``: - UEFI config can now be specified as a parameter. - Adding support for U-Boot booting. - No longer auto-disconnects ADB at the end of a run. - Added ``actually_disconnect`` to restore old disconnect behaviour - Now passes ``video`` command line to Juno kernel to work around a known issue where HDMI loses sync with monitors. - Fixed flashing. Instruments ~~~~~~~~~~~ - ``trace_cmd``: - Fixed ``buffer_size_file`` for non-Android devices - Reduce starting priority. - Now handles trace headers and thread names with spaces - ``energy_probe``: Added ``device_entry`` parameter. - ``hwmon``: - Sensor discovery is now done only at the start of a run. - Now prints both before/after and mean temperatures. - ``daq``: - Now reports energy - Fixed file descriptor leak - ``daq_power.csv`` now matches the order of labels (if specified). - Added ``gpio_sync``. When enabled, this wil cause the instrument to insert a marker into ftrace, while at the same time setting a GPIO pin high. - Added ``negative_values`` parameter. which can be used to specify how negative values in the samples should be handled. - Added ``merge_channels`` parameter. When set DAQ channel will be summed together. - Workload labels, rather than names, are now used in the "workload" column. - ``cpufreq``: - Fixes missing directories problem. - Refined the availability check not to rely on the top-level cpu/cpufreq directory - Now handles non-integer output in ``get_available_frequencies``. - ``sysfs_extractor``: - No longer raises an error when both device and host paths are empty. - Fixed pulled files verification. - ``perf``: - Updated binaries. - Added option to force install. - ``killall`` is now run as root on rooted Android devices. - ``fps``: - now generates detailed FPS traces as well as report average FPS. - Updated jank calcluation to only count "large" janks. - Now filters out bogus ``actual-present`` times and ignore janks above ``PAUSE_LATENCY`` - ``delay``: - Added ``fixed_before_start`` parameter. - Changed existing ``*_between_specs`` and ``*_between_iterations`` callbacks to be ``very_slow`` - ``streamline``: - Added Linux support - ``gatord`` is now only started once at the start of the run. modules ~~~~~~~ - ``flashing``: - Fixed vexpress flashing - Added an option to keep UEFI entry Result Processors ~~~~~~~~~~~~~~~~~ - ``cpustate``: - Now generates a timeline csv as well as stats. - Adding ID to overall cpustate reports. - ``csv``: (partial) ``results.csv`` will now be written after each iteration rather than at the end of the run. Workloads ~~~~~~~~~ - ``glb_corporate``: clears logcat to prevent getting results from previous run. - ``sysbench``: - Updated sysbench binary to a statically linked verison - Added ``file_test_mode parameter`` - this is a mandatory argumet if ``test`` is ``"fileio"``. - Added ``cmd_params`` parameter to pass options directily to sysbench invocation. - Removed Android browser launch and shutdown from workload (now runs on both Linux and Android). - Now works with unrooted devices. - Added the ability to run based on time. - Added a parameter to taskset to specific core(s). - Added ``threads`` parameter to be consistent with dhrystone. - Fixed case where default ``timeout`` < ``max_time``. - ``Dhrystone``: - added ``taskset_mask`` parameter to allow pinning to specific cores. - Now kills any running instances during setup (also handles CTRL-C). - ``sysfs_extractor``: Added parameter to explicitly enable/disable tempfs caching. - ``antutu``: - Fixed multi-``times`` playback for v5. - Updated result parsing to handle Android M logcat output. - ``geekbench``: Increased timout to cater for slower devices. - ``idle``: Now works on Linux devices. - ``manhattan``: Added ``run_timemout`` parameter. - ``bbench``: Now works when binaries_directory is not in path. - ``nemamark``: Made duration configurable. Framework ~~~~~~~~~~ - ``BaseLinuxDevice``: - Now checks that at least one core is enabled on another cluster before attempting to set number of cores on a cluster to ``0``. - No longer uses ``sudo`` if already logged in as ``root``. - Now saves ``dumpsys window`` output to the ``__meta`` directory. - Now takes ``password_prompt`` as a parameter for devices with a non standard ``sudo`` password prompt. - No longer raises an error if ``keyfile`` or ``password`` are not provided when they are not necessary. - Added new cpufreq APIs: - ``core`` APIs take a core name as the parameter (e.g. "a15") - ``cluster`` APIs take a numeric cluster ID (eg. 0) - ``cpu`` APIs take a cpufreq cpu ID as a parameter. - ``set_cpu_frequency`` now has a ``exact`` parameter. When true (the default) it will produce an error when the specified frequency is not supported by the cpu, otherwise cpufreq will decide what to do. - Added ``{core}_frequency`` runtime parameter to set cluster frequency. - Added ``abi`` property. - ``get_properties`` moved from ``LinuxDevice``, meaning ``AndroidDevice`` will try to pull the same files. Added more paths to pull by default too. - fixed ``list_file_systems`` for Android M and Linux devices. - Now sets ``core_clusters`` from ``core_names`` if not explicitly specified. - Added ``invoke`` method that allows invoking an executable on the device under controlled contions (e.g. within a particular directory, or taskset to specific CPUs). - No longer attempts to ``get_sysfile_value()`` as root on unrooted devices. - ``LinuxDevice``: - Now creates ``binaries_directory`` path if it doesn't exist. - Fixed device reset - Fixed ``file_exists`` - implemented ``get_pid_of()`` and ``ps()``. Existing implementation relied on Android version of ps. - ``listdir`` will now return an empty list for an empty directory instead of a list containing a single empty string. - ``AndroidDevice``: - Executable (un)installation now works on unrooted devices. - Now takes into account ``binar_directory`` when setting up busybox path. - update ``android_prompt`` so that it works even if is not ``"/"`` - ``adb_connect``: do not assume port 5555 anymore. - Now always deploys busybox on rooted devices. - Added ``swipe_to_unlock`` method. - Fixed initialization of ``~/.workload_automation.``. - Fixed replaying events using revent on 64 bit platforms. - Improved error repoting when loading extensions. - ``result`` objects now track their output directories. - ``context.result`` will not result in ``context.run_result`` when not executing a job. - ``wlauto.utils.ssh``: - Fixed key-based authentication. - Fixed carriage return stripping in ssh. - Now takes ``password_prompt`` as a parameter for non standard ``sudo`` password prompts. - Now with 100% more thread safety! - If a timeout condition is hit, ^C is now sent to kill the current foreground process and make the shell available for subsequent commands. - More robust ``exit_code`` handling for ssh interface - Now attempts to deal with dropped connections - Fixed error reporting on failed exit code extraction. - Now handles backspaces in serial output - Added ``port`` argument for telnet connections. - Now allows telnet connections without a password. - Fixed config processing for extensions with non-identifier names. - Fixed ``get_meansd`` for numbers < 1 - ``wlatuo.utils.ipython``: - Now supports old versions of IPython - Updated version check to only initialize ipython utils if version is < 4.0.0. Version 4.0.0 changes API and breaks WA's usage of it. - Added ``ignore`` parameter to ``check_output`` - Agendas: - Now raise an error if an agenda contains duplicate keys - Now raise an error if config section in an agenda is not dict-like - Now properly handles ``core_names`` and ``core_clusters`` - When merging list parameters from different sources, duplicates are no longer removed. - The ``INITIAL_BOOT`` signal is now sent went performing a hard reset during intial boot - updated ``ExecutionContext`` to keep a reference to the ``runner``. This will enable Extenstions to do things like modify the job queue. - Parameter now automatically convert int and boot kinds to integer and boolean respectively, this behavior can be supressed by specifying ``convert_types``=``False`` when defining the parameter. - Fixed resource resolution when dependency location does not exist. - All device ``push`` and ``pull`` commands now raise ``DeviceError`` if they didn't succeed. - Fixed showing Parameter default of ``False`` for boolean values. - Updated csv result processor with the option to use classifiers to add columns to ``results.csv``. - ``wlauto.utils.formatter``: Fix terminal size discovery. - The extension loader will now follow symlinks. - Added arm64-v8a to ABI map - WA now reports syntax errors in a more informative way. - Resource resolver: now prints the path of the found resource to the log. - Resource getter: look for executable in the bin/ directory under resource owner's dependencies directory as well as general dependencies bin. - ``GamingWorkload``: - Added an option to prevent clearing of package data before execution. - Added the ability to override the timeout of deploying the assets tarball. - ``ApkWorkload``: Added an option to skip host-side APK check entirely. - ``utils.misc.normalize``: only normalize string keys. - Better error reporting for subprocess.CalledProcessError - ``boolean`` now interprets ``'off'`` as ``False`` - ``wlauto.utils.uefi``: Added support for debug builds. - ``wlauto.utils.serial_port``: Now supports fdexpect versions > 4.0.0 - Semanatics for ``initialize``/``finalize`` for *all* Extensions are changed so that now they will always run at most once per run. They will not be executed twice even if invoked via instances of different subclasses (if those subclasses defined their own verions, then their versions will be invoked once each, but the base version will only get invoked once). - Pulling entries from procfs does not work on some platforms. WA now tries to cat the contents of a property_file and write it to a output file on the host. Documentation ~~~~~~~~~~~~~ - ``installation``: - Added ``post install`` section which lists workloads that require additional external dependencies. - Added the ``uninstall`` and ``upgrade`` commands for users to remove or upgrade Workload Automation. - Added documentation explaining how to use ``remote_assets_path`` setting. - Added warning about potential permission issues with pip. - ``quickstart``: Added steps for setting up WA to run on Linux devices. - ``device_setup``: fixed ``generic_linux`` ``device_config`` example. - ``contributing``: Clarified style guidelines - ``daq_device_setup``: Added an illustration for DAQ wiring. - ``writing_extensions``: Documented the Workload initialize and finalize methods. - Added descriptions to extension that didn't have one. Other ~~~~~ - ``daq_server``: - Fixed showing available devices. - Now works with earlier versions of the DAQmx driver.thus you can now run the server on Linux systems. - DAQ error messages are now properly propaged to the client. - Server will now periodically clean up uncollected files. - fixed not being able to resolve IP address for hostname (report "localhost" in that case). - Works with latest version of twisted. - ``setup.py``: Fixed paths to work with Mac OS X. - ``summary_csv`` is no longer enabled by default. - ``status`` result processor is now enabled by default. - Commands: - ``show``: - Now shows what platform extensions support. - Will no longer try to use a pager if ``PAGER=''`` in the environment. - ``list``: - Added ``"-p"`` option to filter results by supported platforms. - Added ``"--packaged-only"`` option to only list extensions packaged with WA. - ``run``: Added ``"--disable"`` option to diable instruments. - ``create``: - Added ``agenda`` sub-command to generate agendas for a set of extensions. - ``create workload`` now gives more informative errors if Android SDK installed but no platform has been downloaded. Incompatible changes #################### Framework ~~~~~~~~~ - ``BaseLinuxDevice``: - Renamed ``active_cpus`` to ``online_cpus`` - Renamed ``get_cluster_cpu`` to ``get_cluster_active_cpu`` - Renamed ``get_core_cpu`` to ``get_core_online_cpu`` - All extension's ``initialize`` function now takes one (and only one) parameter, ``context``. - ``wlauto.core.device``: Removed ``init`` function. Replaced with ``initialize`` ------------- Version 2.3.0 ------------- - First publicly-released version.