# CLI Usage
Although this section primarily targets users, developers will likely find it helpful as well. The purpose of this section is twofold:
- To familiarize you with the built-in features of the OpenTAP CLI, and get started installing plugins.
- To introduce you to useful tools for running test plans.
Since the core value of OpenTAP comes from its extensibility through plugins, the application itself ships with a few essential components:
- A package manager to browse and install packages.
- The capability to execute test plans.
This keeps the core engine fast, lean, and enables easy deployment in container solutions such as Docker. The CLI help of a clean OpenTAP installation looks something like this:
> tap OpenTAP Command Line Interface (9.9.0) Usage: tap <command> [<subcommand(s)>] [<args>] Valid commands are: run Run a test plan. package create Create a package based on an XML description file. download Download one or more packages. install Install one or more packages. list List locally installed packages and browse the online package repository. show Show information about a package. test Run tests on one or more packages. uninstall Uninstall one or more packages. verify Verify the integrity of one or all installed packages by checking their fingerprints. sdk gitversion Calculate the semantic version number for a specific git commit. Run "tap.exe <command> [<subcommand>] -h" to get additional help for a specific command.
sdk subcommand is targeted at developers, and will not be covered in this section.
Every CLI action, whether built-in or user-provided, shares three CLI options:
| ||Output help text for the given command|
| ||Send all debug-level log messages to standard output. The additional output shown here is always available in the session logs|
| ||Color standard output messages according to their severity|
# Using the Package Manager
The package manager is meant for installing, uninstalling, and creating packages containing plugins. It is capable of listing the available packages and their versions based on the CPU architecture and the operating system on which it is running. Package names are case sensitive, and package names containing spaces, such as "Developer's System CE", must be surrounded by quotation marks.
The package manager has several subcommands, which you can see by running
> tap package OpenTAP Command Line Interface (9.9.0) Usage: tap <command> [<subcommand(s)>] [<args>] Valid subcommands of package create Create a package based on an XML description file. download Download one or more packages. install Install one or more packages. list List locally installed packages and browse the online package repository. show Show information about a package. test Run tests on one or more packages. uninstall Uninstall one or more packages. verify Verify the integrity of one or all installed packages by checking their fingerprints. Run "tap.exe <command> [<subcommand>] -h" to get additional help for a specific command.
test options are geared towards developers, and will not be covered in this section.
The OpenTAP package manager assumes semantic versioning (opens new window) is honored in the likely event dependency resolution is needed, and OpenTAP itself uses semantic versioning.
There are a few CLI options, which most
package subcommands have in common:
| ||Override the OS (Linux, Windows) to target|
| ||Override the CPU architecture (x86, x64, AnyCPU) to target|
| ||Override the repository to look for packages in (it can be a URL, IP address, file path, or a network drive)|
| ||Override the directory where the command is applied. The default is the OpenTAP installation directory.|
The default values of
architecture are automatically configured according to the machine where OpenTAP is
installed, and the default repository is the official OpenTAP repository, packages.opentap.io (opens new window).
By default, all package commands apply operations in the OpenTAP installation directory, where the
tap.exe file is located. The
option makes it possible to manage multiple
tap versions on the same machine.
This is usually what you want, but there are some situations where it may be useful to modify them. For example, you can
install OpenTAP onto a Linux machine from a Windows machine with
tap package install OpenTAP --os Linux --target C:\path\to\linux\installation.
list command is used to view information about packages in your local OpenTAP installation, and available packages
in the online repository.
In order to check what packages are available, run
tap package list. To see what versions of a package are available,
such as OpenTAP itself, run
tap package list OpenTAP.
To see what packages are currently installed, use the
tap package list --installed option. You can view what
packages are in a specific installation directory with
tap package list --installed --target <installation dir>.
list only shows packages compatible with your OS and CPU architecture, and your currently installed
packages. To see also packages incompatible with your OS and CPU architecture, use the
If a version is specified,
list displays all sub-versions of that version.
tap package list OpenTAP --version 9.4
lists OpenTAP versions 9.4.0, 9.4.1, and 9.4.2.
Show details about a package. If
--offline is specified, OpenTAP will only search the local installation for
--include-plugins display all files and plugins included in the package:
tap package show OpenTAP --include-files.
The basic usage of
install is quite simple, but there are options for advanced usage that you may find interesting.
Before moving on to esoteric usage, a few pitfalls must be clarified:
You can install multiple packages at the same time.
tap package install OpenTAP "Editor CE".
Package names sometimes contain spaces. If they do, they must be quoted when referenced in any CLI action, as shown above. Without the quotes, the package manager will interpret
Editor CEas two different package names.
When running the install action, the package manager looks in the
<installation dir>/PackageCachedirectory first, and then in the specified repository.
You can install a local .TapPackage file, acquired with the
downloadsubcommand for example, with
tap package install <filename>.
By default, the
install action installs the latest release of a given package for your platform. Updating any package,
including OpenTAP itself, is easy. Just run
tap package install <package>.
Installing a specific version of any package is also simple:
tap package install OpenTAP --version 9.5.1 installs version 9.5.1;
... --version beta installs the latest beta;
... --version rc installs the latest release candidate.
Whenever you install a package, the package manager will attempt to resolve the dependencies. If you are missing a
package dependency, the package manager will prompt you, and install it automatically if you confirm. To avoid this
behavior, you may install a package with the
-y option to automatically confirm all prompts. If you are trying to
install a package that is incompatible with your current installation, the package manager will stop. This could happen if
you have a package installed that depends on OpenTAP >= 9.5.1, and you try to install OpenTAP 9.4. You can override this
behavior by using the
--force option, but this can lead to a non-functional installation.
--architecture, you can install packages built for different operating systems and architectures. If
you specify values different from your system, they will likely not work. However, used in conjunction with the
--target option, you can use them to install packages on a different machine. The
--target option allows you to
specify an installation directory. This creates a new tap installation in the specified directory with only the plugins
required for the packages you requested. This could be useful, for instance, if you need to install a package that is
incompatible with your current tap installation. You can also install a different version of OpenTAP in another location
tap package install OpenTAP --version 9.4.2 --target C:\path\to\other\installation.
New plugins may provide their own CLI actions, thus increase the number of options. OpenTAP keeps track of the installed
plugins for you, so you can always verify the available CLI actions by running
Uninstall one or more packages, e.g.,
tap package uninstall Demonstration Python. The package manager will warn you if you attempt to uninstall a package other packages in
your installation depend on. Uninstalling dependencies in spite of warnings may break your installation. However, unless
you removed OpenTAP, you can repair your installation by reinstalling the uninstalled dependency.
Like the above two commands,
uninstall supports targeting a different directory.
Download one or more packages without installing them. Dependencies can be automatically downloaded by including the
All downloaded files are placed in the OpenTAP installation directory. You can specify a different destination using the
If it points to a directory, all downloaded packages are placed in that directory instead.
If it specifies a filename, this will be the name of the first package specified, and the remaining packages will be placed in the same directory with their default name.
Like the install action, the download action also supports specifying the desired os, version, and architecture.
tap package download Python.
Verify the integrity of a given package by computing the fingerprints of its locally installed files, and comparing them
with the fingerprints stored in the local XML package description file (
If no package name is provided, all installed packages are verified.
This only works when run from the OpenTAP installation directory.
# Running Test Plans
run commands executes a test plan.
> tap run -h Usage: run [-h] [-v] [-c] [--settings <arg>] [--search <arg>] [--metadata <arg>] [--non-interactive] [-e <arg>] [-t <arg>] [--list-external-parameters] [--results <arg>] <Test Plan> -h, --help Write help information. -v, --verbose Show verbose/debug-level log messages. -c, --color Color messages according to their severity. --settings Specify a bench settings profile from which to load the bench settings. The parameter given here should correspond to the name of a subdirectory of <OpenTAP installation dir>/Settings/Bench. If not specified, <OpenTAP installation dir>/Settings/Bench/Default is used. --search Additional directories to be searched for plugins. This option may be used multiple times, e.g., --search dir1 --search dir2. --metadata Set a resource metadata parameter. Use the syntax parameter=value, e.g., --metadata dut-id=5. This option may be used multiple times. --non-interactive Never prompt for user input. -e, --external Set an external test plan parameter. Use the syntax parameter=value, e.g., -e delay=1.0. This option may be used multiple times, or a .csv file containing a "parameter, value" pair on each line can be specified as -e file.csv. -t, --try-external Try setting an external test plan parameter, ignoring errors if it does not exist in the test plan. Use the syntax parameter=value, e.g., -t delay=1.0. This option may be used multiple times. --list-external-parameters List the available external test plan parameters. --results Enable a subset of the currently configured result listeners given as a comma-separated list, e.g., --results SQLite,CSV. To disable all result listeners use --results "".
# Bench Settings
Specify a bench settings profile for the test plan being run. This refers to the configuration of DUTs, connections, and
--settings parameter should be the name of a subdirectory of
If not specified,
<installation dir>/Settings/Bench/Default is used:
tap run MyTestPlan.TapPlan --settings RadioTestSetup.
# Plugin Search Path
By default, OpenTAP searches for plugins in the installation directory and in the
<installation dir>/Packages directory.
More directories to be searched for plugins may be provided with multiple occourrences of the
one for each additional directory:
tap run MyTestPlan.TapPlan --search C:\Users\Me\MyDut --search C:\Users\Me\MyInstrument.
# Result Listeners
When running a test plan, OpenTAP enables by default all configured result listeners.
A subset of them may be enabled by giving their names in a comma-separated list in the
tap run MyTestPlan.TapPlan --results SQLite,CSV.
To disable all result listeners use
# Non-Interactive Mode
Never prompt for user input:
tap run MyTestPlan.TapPlan --non-interactive.
# External Settings
Test step settings can be parameterized on a parent. This way, multiple child test steps in a test plan can share the same setting, which can then be configured in one place.
Because the test plan itself is the parent of all test steps, settings can naturally be parameterized on that as well. Settings that are parameterized on the test plan are considered external, and can be modified from outside the test plan. This makes it possible to reuse the same test plans and easily run them with different parameters.
Imagine the following testplan:
Test Plan Delay -> Parameterize Time Delay on TestPlan as 'Parameters \ Time Delay 1' Delay -> Parameterize Time Delay on TestPlan as 'Parameters \ Time Delay 2' Delay -> Parameterize Time Delay on TestPlan as 'Parameters \ Time Delay 3'
Let's see how these external settings can be modified.
# Modifying settings from the CLI
To see what external parameters the plan contains, run:
tap run VariableDelay.TapPlan --list-external-parameters
This should output something like:
Test Plan: VariableDelay Listing 3 External Test Plan Parameters: Parameters \ Time Delay 1 = 0.1 s Parameters \ Time Delay 2 = 0.1 s Parameters \ Time Delay 3 = 0.1 s
Here we see they all have the default value
0.1 s. Those values can be overriden when running the
test plan with the
tap run VariableDelay.TapPlan -e "Parameters \ Time Delay 1=0.2" -e "Parameters \ Time Delay 2=0.3"
To verify that the parameters are set correctly,
--list-external-parameters can be used in conjunction with
tap run VariableDelay.TapPlan -e "Parameters \ Time Delay 1=0.2" -e "Parameters \ Time Delay 2=0.3" --list-external-parameters
Test Plan: VariableDelay Listing 3 External Test Plan Parameters: Parameters \ Time Delay 1 = 0.2 s Parameters \ Time Delay 2 = 0.3 s Parameters \ Time Delay 3 = 0.1 s
Note that each parameter along with its assignment and value are quoted. This is required if the parameter name or the assigned value contains a space.
In addition, there must be no space between the assignment operator (
=) and the assigned value. In addition, units such as
s in this case, should
Alternatively, if the CSV (opens new window) plugin is installed,
you can create a csv file named
MyExternalParameters.csv containing the parameters and values:
Parameters \ Time Delay 1,0.7 Parameters \ Time Delay 2,1.4 Parameters \ Time Delay 3,2.1
and then use it with:
tap run VariableDelay.TapPlan -e MyExternalParameters.csv.
# Modifying settings from a Test Plan Reference
If a test plan with external parameters is used in a
Test Plan Reference step,
its parameters will appear as regular step settings, and they can be easily modified.
This means that a collection of test steps can easily be abstracted away to appear as a single,
fully configurable test step.
Similarly to External settings, resource settings can be marked as Metadata. This could be the serial number of a DUT, for instance. This option may be specified multiple times:
tap run My.TapPlan --metadata dut1=123 --metadata dut2=456.