Introduction

The OpenTAP Runner allows for remote starting and control of OpenTAP sessions via NATS. These sessions can create, load, and execute test plans, modify test plan and test step properties, adjust OpenTAP settings, and provide logs and results.

When to use the OpenTAP Runner?

You should consider the OpenTAP Runner if:

• Your solution requires remote interactions with Test Automation
• Your solution is implemented in another language than C#
• You want to integrate with KS8500

Concepts

OpenTAP Runner Communication

The OpenTAP Runner communicates using NATS.io, a high-performance messaging system that provides a simple yet powerful API for building distributed systems and applications.

The NATS Server can be an instance that you control, or you can install the NATS Server TapPackage to let OpenTAP Runner start the NATS Server, which is the recommended approach.

To install the NATS Server package, use the following command: tap package install "NATS Server"

After installation, the runner can be started with the command: tap runner start

The Runner can start Runner Sessions. A Runner Session is a process that also communicates on NATS and exposes the APIs to conduct the same functionality as KS8400A PathWave Test Automation. This includes creating, loading, modifying, and executing test plans.

There are two ways to start a Runner Session through the Runner: using an Image or just a simple copy of the Runner installation. An Image is a set of OpenTAP Packages and a set of OpenTAP Repositories from where to fetch the packages.

The simple case where we ask the Runner to start a Runner Session not based on an Image looks like this:

The more remote-friendly and flexible solution is to use Images. If you require a Session with a different set of packages than the current Runner installation provides, create an Image in the Runner and then use that Image to start a Runner Session:

Getting started

This guide provides a quick tutorial on setting up and running an OpenTAP project using the OpenTAP Runner and NATS Server. The tutorial will walk you through installing necessary packages, setting up a basic C# Console project, and interacting with the Runner.

Compatibility Runner Client 3.x targets the asynchronous API and is wire-compatible with Runner 2.x. Use Runner Client 2.x only when talking to Runner 1.x installations.

Prerequisites

• An OpenTAP installation

Starting the Runner

In an OpenTAP installation, install the two OpenTAP Packages, "Runner" and "NATS Server":

./tap image install "Runner,NATS Server" --merge

Then start the Runner to start the OpenTAP Runner and the NATS Server:

./tap runner start

The NATS Server process will by default start on nats://127.0.0.1:20111 and the Runner will listen on the OpenTap.Runner.{MachineName} subject.

Docker images

OpenTAP Runner publishes two Linux images to GHCR for Keysight teams:

• Static (version-tagged) image: ghcr.io/opentap/runner:<version>
• Updatable image: ghcr.io/opentap/runner:updatable (and :updatable-beta / :updatable-rc for prereleases)

Static images keep the OpenTAP installation inside the image and only expose data folders as volumes. Updatable images mount the full /opt/tap installation so in-container updates persist across restarts.

Example: run a static image and persist data

docker run --rm -p 20111:20111 -p 20116:20116 \
  -v runner-storage:/opt/tap/Storage \
  -v runner-logs:/opt/tap/SessionLogs \
  -v runner-settings:/opt/tap/Settings \
  ghcr.io/opentap/runner:<version>

Example: run the updatable image

docker run --rm -p 20111:20111 -p 20116:20116 \
  -v runner-installation:/opt/tap \
  ghcr.io/opentap/runner:updatable

These images are currently only available to Keysight teams. If you are outside Keysight and are interested in access, reach out via the contact information on the contact page.

Creating a C# Console Project

1. Create a new C# Console project:

mkdir MyApp
cd MyApp
dotnet new console

2. Add the Runner Client package:

dotnet add package OpenTAP.Runner.Client --version 3.*

3. Modify Program.cs to interact with the Runner. The Runner Client 3.x surface is asynchronous, so we use top-level await (available in recent .NET templates):

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runnerClient = new RunnerClient(connection);
SessionClient sessionClient = await runnerClient.StartSessionAsync();

sessionClient.ConnectSessionLogs(handleSessionLogs);

static Task handleSessionLogs(LogList? logList)
{
    if (logList?.Logs == null)
        return Task.CompletedTask;

    foreach (var log in logList.Logs)
        Console.WriteLine($"{log.Source,-12}: {log.Message}");

    return Task.CompletedTask;
}

// Get the available step types (Similar to "Add new step" dialog)
List<TestStepType> stepTypes = await sessionClient.GetStepTypesAsync();

// Get the current test plan from the session
TestPlan testPlan = await sessionClient.GetTestPlanAsync(null);

// Add a delay step to the test plan
testPlan.ChildTestSteps.Add(stepTypes.First(step => step.TypeName.Contains("Delay")));

// Update the test plan on the session
testPlan = await sessionClient.SetTestPlanAsync(testPlan);

// Give the test plan a name
await sessionClient.SetTestPlanNameAsync("My Testplan");

// Start the test plan
RunStatus runStatus = await sessionClient.RunTestPlanAsync(new List<Parameter>());

// Wait for the test plan to finish
while (runStatus.SessionState != SessionState.Idle)
{
    await Task.Delay(250);
    runStatus = await sessionClient.GetStatusAsync();
}

// Shutdown the session
await sessionClient.ShutdownAsync();
await runnerClient.ShutdownSessionAsync(sessionClient.Id);

This code connects, starts a Session, connects to session logs, creates a test plan and executes it. After execution finishes, the session is closed.

4. Run the project:

dotnet run

The expected output of dotnet run:

TestPlan    : -----------------------------------------------------------------
TestPlan    : Starting TestPlan 'My Testplan' on 04/05/2023 14:36:25, 1 of 1 TestSteps enabled.
TestPlan    : Saved Test Plan XML [1.66 ms]
Settings    : No settings file exists for TestPlanRunPackageParameterMonitor. A new instance with default values has been created. [200 us]
TestPlan    : PrePlanRun Methods completed [2.46 ms]
TestPlan    : "Delay" started.
TestPlan    : "Delay" completed. [107 ms]
TestPlan    : Test step runs finished. [119 ms]
Summary     : ----- Summary of test plan started 04/05/2023 14:36:25 -----
Summary     :  Delay                                             107 ms
Summary     : ------------------------------------------------------------
Summary     : -------- Test plan completed successfully in 199 ms --------

With these steps, you have successfully set up and run an application that remotely executed a testplan using the OpenTAP Runner and NATS Server.

We can now stop the OpenTAP Runner (and NATS Server) by sending CTRL+C in the command prompt with the Runner.

Code Examples

Compatibility Runner Client 3.x exposes asynchronous APIs and is the recommended client for Runner 2.x servers.

1. Working with Images and starting Sessions based on Images
2. Simple session creation based on Runner installation
3. Create and execute a test plan
4. Editing step and test plan settings
5. Load and execute a test plan
6. Receiving results, logs and events from a Session
7. How to work with component settings
8. Configuring a Runner for default execution and using stored test plans on the Runner
9. Copy, paste, undo, and redo test steps
10. Working with UserInputs
11. Debugging: Breakpoints, pause, and jump-to-step
12. Session watchdog configuration
13. Test plan validation errors
14. Runner status and health monitoring
15. Moving and deleting test steps
16. Querying and searching session logs

Image creation

For this example we have a Runner started which has the ID: HKZ6QN3

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);

Image image = new Image
{
    Packages = new List<PackageSpecifier>
    {
        new PackageSpecifier { Name = "Demonstration" },
        new PackageSpecifier { Name = "HTTP", Version = "1.0.0" },

        // We can even specify a different runner version for the session
        new PackageSpecifier { Name = "Runner", Version = "2" }
    },
    Repositories = new List<string> { "https://packages.opentap.io" }
};

Image? resolvedImage = await runner.ResolveImageAsync(new List<Image> { image });
if (resolvedImage is null)
    throw new InvalidOperationException("Image resolution failed.");

SessionClient newSession = await runner.StartImageSessionAsync(resolvedImage);

// Use the session for test automation

await runner.ShutdownSessionAsync(newSession.Id);

In this example, we successfully resolved an image, started and stopped a session based on the image. This can be verified in the Runner logs:

Session creation

Although the Image approach offers flexibility, some use-cases can be kept simple by just using a Runner Session based on the Runner installation.

using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);

SessionClient newSession = await runner.StartSessionAsync();

// Use the session for test automation. Plugins available in this session
// come from the packages installed in the Runner.

await runner.ShutdownSessionAsync(newSession.Id);

Create and Run Test Plan

In this example, we use the simple session from the Session creation example above, but before shutting down the session, we setup and run a simple test plan.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient newSession = await runner.StartSessionAsync();

List<TestStepType> types = await newSession.GetStepTypesAsync();
TestPlan plan = await newSession.GetTestPlanAsync(null);
plan.ChildTestSteps.Clear();
plan.ChildTestSteps.Add(types.First(step => step.TypeName.Contains("Delay")));
plan = await newSession.SetTestPlanAsync(plan);

Settings delayStepSettings = await newSession.GetSettingsAsync(plan.ChildTestSteps[0].Id);
if (delayStepSettings.FirstOrDefault(s => s.PropertyName == "DelaySecs") is TextBoxControl textBox)
    textBox.StringValue = "3 s";
delayStepSettings = await newSession.SetSettingsAsync(plan.ChildTestSteps[0].Id, delayStepSettings);

RunStatus status = await newSession.RunTestPlanAsync(new List<Parameter>());
while (status.SessionState != SessionState.Idle)
{
    await Task.Delay(1000);
    status = await newSession.GetStatusAsync();
}

Console.WriteLine(status.Verdict);

await runner.ShutdownSessionAsync(newSession.Id);

In this example, we retrieved the available test step types using GetStepTypes and chose a Delay step and inserted into the Test Plan using SetTestPlan.

Afterwards, we retrieved the Delay step settings and modified the Time Delay to be 3 s.

Lastly, we ran the Test Plan and continually asked for the status until the SessionState turned back to Idle. The Verdict of the Test Plan was logged and the output of this program would be NotSet.

Editing Step and TestPlan Settings

In this example, we add two steps to a test plan and demonstrate how to read and modify settings on both steps and the plan itself.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient session = await runner.StartSessionAsync();

try
{
    // Add a DelayStep and a LogStep to the test plan
    List<TestStepType> types = await session.GetStepTypesAsync();
    TestStepType delayType = types.First(step => step.TypeName.Contains("Delay"));
    TestStepType logType = types.First(step => step.TypeName.Contains("LogStep"));

    TestPlan plan = await session.GetTestPlanAsync(null);
    plan.ChildTestSteps.Clear();
    plan.ChildTestSteps.Add(delayType);
    plan.ChildTestSteps.Add(logType);
    plan = await session.SetTestPlanAsync(plan);

    Guid delayStepId = plan.ChildTestSteps[0].Id;
    Guid logStepId = plan.ChildTestSteps[1].Id;

    // Read and modify a TextBoxControl on the Delay step
    Settings delaySettings = await session.GetSettingsAsync(delayStepId);
    if (delaySettings.FirstOrDefault(s => s.PropertyName == "DelaySecs") is TextBoxControl textBox)
    {
        Console.WriteLine($"Current delay: {textBox.StringValue}");
        textBox.StringValue = "5 s";
    }
    delaySettings = await session.SetSettingsAsync(delayStepId, delaySettings);

    // Read a CheckBoxControl on the Delay step
    if (delaySettings.FirstOrDefault(s => s.Display.Name == "Enabled") is CheckBoxControl checkBox)
    {
        Console.WriteLine($"Enabled: {checkBox.BoolValue}");
        checkBox.BoolValue = !checkBox.BoolValue; // toggle the value
    }
    delaySettings = await session.SetSettingsAsync(delayStepId, delaySettings);

    // Get plan-level settings using the test plan ID
    Settings planSettings = await session.GetSettingsAsync(plan.Id);
    foreach (Setting setting in planSettings)
        Console.WriteLine($"Plan setting: {setting.Display.Name} ({setting.GetType().Name})");

    // Set the test plan name
    await session.SetTestPlanNameAsync("My Custom Plan");
}
finally
{
    await session.ShutdownAsync();
    await runner.ShutdownSessionAsync(session.Id);
}

GetSettingsAsync and SetSettingsAsync accept a Guid context identifier that can refer to either a test step or the test plan itself. Pass plan.ChildTestSteps[0].Id to configure a step, or plan.Id to inspect and modify plan-level properties.

The settings collection contains polymorphic controls such as TextBoxControl, CheckBoxControl, and DropdownControl. Use C# pattern matching (is TextBoxControl textBox) to handle each control type and access its specific properties — StringValue for text boxes, BoolValue for check boxes, and SelectedIndex / AvailableValues for dropdowns.

Settings are round-tripped: retrieve the current values with GetSettingsAsync, modify the returned control instances in place, then pass the entire collection back to SetSettingsAsync to apply the changes. The returned Settings object reflects the server-side state after the update.

Serialize and Deserialize Settings JSON

If you need to serialize or deserialize raw settings payloads outside request/response methods, use RunnerSerialization.SerializerOptions so polymorphic controls deserialize reliably.

using System.Text.Json;
using OpenTap.Runner.Client;

string rawSettingsJson = """
[
  {
    "StringValue": "5 s",
    "PropertyName": "DelaySecs",
    "ControlType": "TextBoxControl"
  }
]
""";

Settings? parsed = JsonSerializer.Deserialize<Settings>(rawSettingsJson, RunnerSerialization.SerializerOptions);
if (parsed is { Count: > 0 } && parsed[0] is TextBoxControl delay)
    delay.StringValue = "10 s";

JsonSerializerOptions writeOptions = RunnerSerialization.SerializerOptions;
writeOptions.WriteIndented = true;
string updatedJson = parsed == null ? "[]" : JsonSerializer.Serialize(parsed, writeOptions);

Load and Run Test Plan

In this example, we use the simple session from the Session creation example above, but before shutting down the session, we load and run a Test Plan locally stored on the disk.

using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient newSession = await runner.StartSessionAsync();

await newSession.SetTestPlanXMLAsync(await File.ReadAllTextAsync("MyTestPlan.TapPlan"));

RunStatus status = await newSession.RunTestPlanAsync(new List<Parameter>());
while (status.SessionState != SessionState.Idle)
{
    await Task.Delay(1000);
    status = await newSession.GetStatusAsync();
}

Console.WriteLine(status.Verdict);

await runner.ShutdownSessionAsync(newSession.Id);

Results, Logs and Events

In this example, we use the simple session from the Create and Run Test Plan section above, but with results, logs and events logged.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient newSession = await runner.StartSessionAsync();

newSession.ConnectSessionLogs(LogHandler);
Task LogHandler(LogList? list)
{
    if (list?.Logs != null)
        foreach (var log in list.Logs)
            Console.WriteLine($"{log.Source,-12}: {log.Message}");
    return Task.CompletedTask;
}

newSession.ConnectSessionResults(ResultHandler, RunHandler);
Task ResultHandler(Result? result)
{
    if (result != null)
        Console.WriteLine($"Result: {result.Status}");
    return Task.CompletedTask;
}
Task RunHandler(TestRun? run)
{
    if (run != null)
        Console.WriteLine($"Run: {run.Status}");
    return Task.CompletedTask;
}

newSession.ConnectSessionEvents(EventHandler);
Task EventHandler(SessionEvent? evt)
{
    if (evt != null)
        Console.WriteLine($"Event: {evt.GetType().Name}");
    return Task.CompletedTask;
}

List<TestStepType> types = await newSession.GetStepTypesAsync();
TestPlan plan = await newSession.GetTestPlanAsync(null);
plan.ChildTestSteps.Clear();
plan.ChildTestSteps.Add(types.First(step => step.TypeName.Contains("Delay")));
plan = await newSession.SetTestPlanAsync(plan);

Settings delayStepSettings = await newSession.GetSettingsAsync(plan.ChildTestSteps[0].Id);
if (delayStepSettings.FirstOrDefault(s => s.PropertyName == "DelaySecs") is TextBoxControl textBox)
    textBox.StringValue = "3 s";
delayStepSettings = await newSession.SetSettingsAsync(plan.ChildTestSteps[0].Id, delayStepSettings);

RunStatus status = await newSession.RunTestPlanAsync(new List<Parameter>());
while (status.SessionState != SessionState.Idle)
{
    await Task.Delay(1000);
    status = await newSession.GetStatusAsync();
}

Console.WriteLine(status.Verdict);

await runner.ShutdownSessionAsync(newSession.Id);

This code should produce the following output, where events, runs and results are prepended with "Event: ", "Run: ", and "Result: ".

Working with ComponentSettings

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient newSession = await runner.StartSessionAsync();

try
{
    List<ComponentSettingsIdentifier> componentSettings = await newSession.GetComponentSettingsOverviewAsync();
    foreach (var componentSetting in componentSettings)
        Console.WriteLine($"Component Setting: {componentSetting.Name} ({componentSetting.GroupName})");

    // Outputs:
    // Component Setting: DUTs(Bench)
    // Component Setting: Engine(-)
    // Component Setting: Instruments(Bench)
    // Component Setting: Connections(Bench)
    // Component Setting: Results(-)
    // Component Setting: Editor(-)
    // Component Setting: Cloud Drive(-)

    ComponentSettingsIdentifier instrumentIdentifier = componentSettings
        .First(s => s.Name == "Instruments" && s.GroupName == "Bench");

    List<ListItemType> instrumentTypes = await newSession.GetComponentSettingsListAvailableTypesAsync(
        instrumentIdentifier.GroupName,
        instrumentIdentifier.Name);
    foreach (var instrumentType in instrumentTypes)
        Console.WriteLine($"Instrument Type: {string.Join("/", instrumentType.TypeDisplay.Group)}/{instrumentType.TypeDisplay.Name}");

    // Outputs:
    // Instrument Type: / Generic SCPI Instrument
    // Instrument Type: Demo / Battery Test / Power Analyzer
    // Instrument Type: Demo / Battery Test / Temperature Chamber
    // Instrument Type: Demo / Results And Timing/ Timing Instrument

    ComponentSettingsBase instruments = await newSession.GetComponentSettingsAsync(
        instrumentIdentifier.GroupName,
        instrumentIdentifier.Name);
    if (instruments is ComponentSettingsList list)
        foreach (var instrument in list.Items)
            Console.WriteLine($"Instrument: {instrument.Name}");

    // Outputs:
    // Instrument: SCPI
    // Instrument: PSU
    // Instrument: TEMP

    ComponentSettingsBase updated = await newSession.AddComponentSettingsListItemAsync(
        instrumentIdentifier.GroupName,
        instrumentIdentifier.Name,
        instrumentTypes.First(type => type.TypeDisplay.Name == "Power Analyzer").TypeName);

    if (updated is ComponentSettingsList listWithNewItem)
    {
        foreach (var instrument in listWithNewItem.Items)
            Console.WriteLine($"Instrument: {instrument.Name}");

        // Outputs:
        // Instrument: SCPI
        // Instrument: PSU
        // Instrument: TEMP
        // Instrument: PSU (1)

        foreach (var setting in listWithNewItem.Items[^1].Settings)
        {
            Console.WriteLine($"Setting: {setting.Display.Name}");
            if (setting is TextBoxControl textBox)
                Console.WriteLine(textBox.StringValue);
        }
    }

    // Outputs:
    // Setting: Cell Size Factor
    // 0.005
}
finally
{
    await runner.ShutdownSessionAsync(newSession.Id);
}

Default Endpoints

Default endpoints are endpoints that affect each other. They are:

• SetDefaultImage: Sets a default image in the Runner environment. The image includes essential packages.
• GetDefaultImage: Retrieves the currently set default image.
• SetDefaultSettings: Applies default settings for the Runner. Settings might have various package dependencies.
• GetDefaultSettings: Retrieves the current settings from the Runner.
• StartDefaultSession: Starts a session using the default settings and image.
• StartDefaultSessionOverrideImage: Begins a session with an overridden image for specific use cases.

A Session's plugin configuration depends on:

• The dependencies of the TestPlan.
• The dependencies of the Settings.
• The packages defined in a "default image".
• Optionally, the packages defined in an "image override".

The following example demonstrates how to use the RunnerClient for managing test plans in a Runner session.

public static async Task DefaultTestPlanShowcaseAsync(RunnerClient runnerClient)
{
    Image baseImage = new()
    {
        Packages = new List<PackageSpecifier>
        {
            new PackageSpecifier { Name = "Runner", Version = "1.7.0" }
        },
        Repositories = new List<string>
        {
            "https://packages.opentap.io"
        }
    };
    await runnerClient.Default.SetImageAsync(baseImage); // Apply the base image configuration

    RepositoryPackageReference settingsPackageReference = new()
    {
        Repository = "https://test-automation.pw.keysight.com/api/packages",
        Path = "/users/dennis.rasmussen@keysight.com/BatterySettings.TapPackage",
        Version = "0.2.0"
    };
    await runnerClient.Default.SetSettingsAsync(settingsPackageReference); // Apply settings package

    RepositoryPackageReference testPlanReference = new()
    {
        Repository = "https://test-automation.pw.keysight.com/api/packages",
        Path = "/users/dennis.rasmussen@keysight.com/BatteryPlan.TapPlan",
        Version = "0.1.0-Draft.5"
    };

    SessionClient session = await runnerClient.Default.StartSessionAsync(testPlanReference);

    try
    {
        RunStatus status = await session.RunTestPlanAsync(new List<Parameter>());
        while (status.SessionState != SessionState.Idle)
        {
            await Task.Delay(1000);
            status = await session.GetStatusAsync();
        }

        Console.WriteLine($"Verdict: {status.Verdict}");
    }
    finally
    {
        await session.ShutdownAsync();
        await runnerClient.ShutdownSessionAsync(session.Id);
    }
}

Copy, paste, undo, and redo test steps

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);

SessionClient session = await runner.StartSessionAsync();

try
{
    List<TestStepType> stepTypes = await session.GetStepTypesAsync();
    TestStepType delay = stepTypes.First(step => step.TypeName == "OpenTap.Plugins.BasicSteps.DelayStep");

    TestPlan plan = await session.GetTestPlanAsync(null);
    plan.ChildTestSteps.Clear();
    plan = await session.SetTestPlanAsync(plan);

    plan.ChildTestSteps.Add(delay);
    plan = await session.SetTestPlanAsync(plan);
    Guid firstStepId = plan.ChildTestSteps[0].Id;

    plan.ChildTestSteps.Add(delay);
    plan = await session.SetTestPlanAsync(plan);
    Guid targetStepId = plan.ChildTestSteps[1].Id;

    string clipboard = await session.CopyStepsAsync(new HashSet<Guid> { firstStepId });
    await session.PasteStepsAsync(targetStepId, clipboard);

    await session.TestPlanUndoAsync(); // removes the pasted copy
    await session.TestPlanRedoAsync(); // reapplies the paste
}
finally
{
    await session.ShutdownAsync();
    await runner.ShutdownSessionAsync(session.Id);
}

Working with UserInputs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient session = await runner.StartSessionAsync();

try
{
    // Used to wait for the test plan to finish executing
    var readyEvent = new ManualResetEventSlim(false);
    bool planStarted = false;

    // Subscribe to session events to handle user input requests
    session.ConnectSessionEvents(async eve =>
    {
        if (eve == null)
            return;

        Console.WriteLine($"Event: {eve.GetType().Name}");

        // When a test step (e.g. DialogStep) requires operator input,
        // the session raises a UserInputRequestedEvent
        if (eve is UserInputRequestedEvent userInputRequest)
        {
            // Retrieve the full Interaction model for this request
            Interaction interaction = await session.GetUserInputAsync(userInputRequest.RequestId);
            Console.WriteLine($"UserInput requested: {interaction.Title}");

            // Find the visible ButtonsControl in the interaction settings
            if (interaction.Settings.OfType<ButtonsControl>().FirstOrDefault(s => s.VisualStatus.IsVisible) is
                ButtonsControl buttons)
            {
                // Select the first button and invoke it
                buttons.SelectedIndex = 0;
                buttons.InvokeMethod = true;

                // Submit the response — the session resumes execution
                await session.SetUserInputAsync(interaction);
            }
        }

        // Signal when the session returns to Idle after the plan has started
        if (eve is SessionStateChangedEvent executionChange)
        {
            if (executionChange.RunStatus.SessionState == SessionState.Executing)
                planStarted = true;
            if (planStarted && executionChange.RunStatus.SessionState == SessionState.Idle)
                readyEvent.Set();
        }
    });

    // Build a test plan with a DialogStep that will trigger a user input
    List<TestStepType> stepTypes = await session.GetStepTypesAsync();
    TestPlan plan = await session.GetTestPlanAsync(null);
    plan.ChildTestSteps.Add(stepTypes.First(step => step.TypeName.Contains("Dialog")));
    plan = await session.SetTestPlanAsync(plan);

    // Run the test plan — execution blocks until the DialogStep's user input is answered
    await session.RunTestPlanAsync(new List<Parameter>());

    // Wait for the session to return to Idle
    if (!readyEvent.Wait(TimeSpan.FromSeconds(30)))
        Console.WriteLine("Test plan did not finish within 30 seconds.");

    RunStatus status = await session.GetStatusAsync();
    Console.WriteLine($"Verdict: {status.Verdict}");
}
finally
{
    await session.ShutdownAsync();
    await runner.ShutdownSessionAsync(session.Id);
}

User inputs are raised by test steps (such as DialogStep) that require operator interaction during execution. When a step requests input, the session publishes a UserInputRequestedEvent containing a RequestId. Use GetUserInputAsync with this ID to retrieve the full Interaction model.

The Interaction contains a Title, Modal flag, Timeout, and Settings. The settings collection uses the same control types as step settings — TextBoxControl, ButtonsControl, and others — so the same patterns for reading and modifying controls apply here. While the session is waiting for a response, its state is WaitingForUserInput.

To submit a response, set InvokeMethod = true on a ButtonsControl (optionally choosing a button via SelectedIndex) and call SetUserInputAsync. The session resumes execution after the input is answered.

Debugging: Breakpoints, pause, and jump-to-step

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient session = await runner.StartSessionAsync();

try
{
    // Track break events so we know which step the session stopped on
    var breakHit = new ManualResetEventSlim(false);
    Guid breakStepId = Guid.Empty;
    bool planStarted = false;
    var planFinished = new ManualResetEventSlim(false);

    session.ConnectSessionEvents(evt =>
    {
        if (evt is BreakEvent breakEvent)
        {
            breakStepId = breakEvent.StepId;
            breakHit.Set();
        }
        if (evt is SessionStateChangedEvent stateChange)
        {
            if (stateChange.RunStatus.SessionState == SessionState.Executing)
                planStarted = true;
            if (planStarted && stateChange.RunStatus.SessionState == SessionState.Idle)
                planFinished.Set();
        }
        return Task.CompletedTask;
    });

    // Build a plan with three Delay steps
    List<TestStepType> types = await session.GetStepTypesAsync();
    TestStepType delayType = types.First(step => step.TypeName.Contains("Delay"));

    TestPlan plan = await session.GetTestPlanAsync(null);
    plan.ChildTestSteps.Clear();
    plan.ChildTestSteps.Add(delayType);
    plan.ChildTestSteps.Add(delayType);
    plan.ChildTestSteps.Add(delayType);
    plan = await session.SetTestPlanAsync(plan);

    Guid step0 = plan.ChildTestSteps[0].Id;
    Guid step1 = plan.ChildTestSteps[1].Id;
    Guid step2 = plan.ChildTestSteps[2].Id;

    // Set each delay to 0.5 seconds
    foreach (Guid stepId in new[] { step0, step1, step2 })
    {
        Settings s = await session.GetSettingsAsync(stepId);
        if (s.FirstOrDefault(x => x.PropertyName == "DelaySecs") is TextBoxControl tb)
            tb.StringValue = "0.5 s";
        await session.SetSettingsAsync(stepId, s);
    }

    // --- Set breakpoints on the first two steps ---
    await session.SetBreakpointsAsync(new BreakPoints { TestSteps = new List<Guid> { step0, step1 } });

    BreakPoints currentBp = await session.GetBreakpointsAsync();
    Console.WriteLine($"Breakpoints set on {currentBp.TestSteps.Count} steps");

    // Run the plan — it will break on step0
    await session.RunTestPlanAsync(new List<Parameter>());
    breakHit.Wait(TimeSpan.FromSeconds(10));
    Console.WriteLine($"Broke on step: {(breakStepId == step0 ? "step0" : breakStepId.ToString())}");

    // --- Resume execution — it will break on step1 ---
    breakHit.Reset();
    await session.RunTestPlanAsync(new List<Parameter>());
    breakHit.Wait(TimeSpan.FromSeconds(10));
    Console.WriteLine($"Broke on step: {(breakStepId == step1 ? "step1" : breakStepId.ToString())}");

    // --- Use SetPauseNext to single-step to the next step ---
    // SetPauseNext resumes execution and breaks on the very next step
    breakHit.Reset();
    await session.SetPauseNextAsync();
    breakHit.Wait(TimeSpan.FromSeconds(10));
    Console.WriteLine($"PauseNext broke on step: {(breakStepId == step2 ? "step2" : breakStepId.ToString())}");

    // --- Jump back to step0 while in Breaking state ---
    // Clear breakpoints first so step0 doesn't trigger another break
    await session.SetBreakpointsAsync(new BreakPoints { TestSteps = new List<Guid>() });
    await session.SetJumpToStepAsync(step0);

    // --- Abort the run ---
    await session.AbortTestPlanAsync();

    planFinished.Wait(TimeSpan.FromSeconds(10));
    RunStatus status = await session.GetStatusAsync();
    Console.WriteLine($"Final verdict: {status.Verdict}");
}
finally
{
    await session.ShutdownAsync();
    await runner.ShutdownSessionAsync(session.Id);
}

Breakpoints pause execution just before a step runs. Set them with SetBreakpointsAsync by passing a BreakPoints object containing the target step IDs. When a breakpoint triggers, the session enters the Breaking state and publishes a BreakEvent with the StepId that caused the break.

While in Breaking state you can:

• Resume — call RunTestPlanAsync to continue execution until the next breakpoint or completion.
• Step — call SetPauseNextAsync to resume and break on the very next step regardless of breakpoints.
• Jump — call SetJumpToStepAsync to redirect execution to a different step. This only works while in the Breaking state.
• Abort — call AbortTestPlanAsync to cancel the run entirely.

Clear all breakpoints by passing an empty list to SetBreakpointsAsync.

Session watchdog configuration

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient session = await runner.StartSessionAsync();

try
{
    // Read the current watchdog configuration
    WatchDog watchdog = await session.GetWatchDogAsync();
    Console.WriteLine($"Current timeout: {watchdog.TerminationTimeout}s, inactive: {watchdog.InactiveSeconds}s");

    // Set the watchdog to terminate the session after 120 seconds of inactivity
    watchdog.TerminationTimeout = 120;
    watchdog = await session.SetWatchDogAsync(watchdog);
    Console.WriteLine($"Updated timeout: {watchdog.TerminationTimeout}s");

    // InactiveSeconds only counts while the session is Idle and
    // resets whenever a NATS request is received. After a brief wait
    // the counter reflects the time since the last API call.
    await Task.Delay(3000);
    watchdog = await session.GetWatchDogAsync();
    Console.WriteLine($"After 3s idle: {watchdog.InactiveSeconds:F1}s inactive");

    // Disable the watchdog by setting TerminationTimeout to 0
    watchdog.TerminationTimeout = 0;
    watchdog = await session.SetWatchDogAsync(watchdog);
    Console.WriteLine($"Watchdog disabled (timeout={watchdog.TerminationTimeout})");
}
finally
{
    await session.ShutdownAsync();
    await runner.ShutdownSessionAsync(session.Id);
}

The session watchdog monitors idle time and terminates unattended sessions. InactiveSeconds tracks how long the session has been idle — it only increments when the session state is Idle and resets whenever a NATS request is received or a test plan runs.

Set TerminationTimeout to the maximum number of idle seconds before the session is automatically shut down. Setting it to 0 disables the watchdog entirely.

Test plan validation errors

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient session = await runner.StartSessionAsync();

try
{
    // Start with a valid plan — no validation errors expected
    List<TestStepType> types = await session.GetStepTypesAsync();
    TestPlan plan = await session.GetTestPlanAsync(null);
    plan.ChildTestSteps.Clear();
    plan.ChildTestSteps.Add(types.First(step => step.TypeName.Contains("Delay")));
    plan = await session.SetTestPlanAsync(plan);

    TestPlanValidationErrors errors = await session.GetValidationErrorsAsync();
    Console.WriteLine($"Validation errors (valid plan): {errors.Count}");

    // Disable the step — disabled steps are excluded from validation
    Guid delayStepId = plan.ChildTestSteps[0].Id;
    Settings settings = await session.GetSettingsAsync(delayStepId);
    if (settings.FirstOrDefault(s => s.Display.Name == "Enabled") is CheckBoxControl enabledBox)
        enabledBox.BoolValue = false;
    await session.SetSettingsAsync(delayStepId, settings);

    errors = await session.GetValidationErrorsAsync();
    Console.WriteLine($"Validation errors (disabled step): {errors.Count}");

    // Print any validation errors found
    foreach (TestStepValidationError stepError in errors)
    {
        Console.WriteLine($"  Step {stepError.StepId}:");
        foreach (ValidationError ve in stepError.ValidationErrors)
            Console.WriteLine($"    {ve.PropertyName}: {ve.Error}");
    }
}
finally
{
    await session.ShutdownAsync();
    await runner.ShutdownSessionAsync(session.Id);
}

GetValidationErrorsAsync returns a TestPlanValidationErrors collection containing one TestStepValidationError per step that has issues. Each entry has a StepId (string GUID) and a list of ValidationError objects with PropertyName and Error message.

Only enabled steps are validated — disabling a step removes it from the validation results. This is useful for temporarily bypassing steps with missing resource dependencies during plan development.

Runner status and health monitoring

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);

// --- Runner-level status ---
RunnerStatus status = await runner.GetStatusAsync();
Console.WriteLine($"Runner version:   {status.Version}");
Console.WriteLine($"Hostname:         {status.Hostname}");
Console.WriteLine($"IP address:       {status.IpAddress}");
Console.WriteLine($"Running as service: {status.RunningAsService}");
Console.WriteLine($"Running in container: {status.RunningInContainer}");

// NATS connection info
if (status.RemoteConnection is RemoteConnection rc)
{
    Console.WriteLine($"NATS endpoint:    {rc.Endpoint}");
    Console.WriteLine($"NATS connected:   {rc.Connected}");
    Console.WriteLine($"NATS RTT:         {rc.Rtt}");
}

// Storage and stream health
if (status.StreamInfo is StreamInfoStatus si)
{
    Console.WriteLine($"Runs max bytes:   {si.RunsMaxBytes}");
    Console.WriteLine($"Runs usage:       {si.RunsUsage}");
    Console.WriteLine($"Disk free:        {si.DiskFree}");
}

// --- Active sessions ---
List<Session> sessions = await runner.GetSessionsAsync();
Console.WriteLine($"\nActive sessions: {sessions.Count}");
foreach (Session s in sessions)
{
    Console.WriteLine($"  {s.Id} — State: {s.SessionState}, StartedBy: {s.StartedBy}");
    foreach (var kv in s.Metadata)
        Console.WriteLine($"    {kv.Key}: {kv.Value}");
}

GetStatusAsync on the RunnerClient returns a RunnerStatus with version, hostname, IP address, and flags indicating whether the Runner is running as a service or in a container. It also includes RemoteConnection (NATS connectivity) and StreamInfoStatus (JetStream storage usage and free disk space) for health monitoring.

GetSessionsAsync lists all active sessions with their state, the user who started them, and any attached metadata. This is useful for building dashboards or monitoring tools that need to track Runner utilization.

Moving and deleting test steps

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient session = await runner.StartSessionAsync();

try
{
    List<TestStepType> types = await session.GetStepTypesAsync();
    TestStepType delayType = types.First(step => step.TypeName.Contains("Delay"));
    TestStepType repeatType = types.First(step => step.TypeName.Contains("RepeatStep"));

    // Build a plan: [Delay A, Delay B, Repeat C]
    TestPlan plan = await session.GetTestPlanAsync(null);
    plan.ChildTestSteps.Clear();
    plan.ChildTestSteps.Add(delayType);
    plan.ChildTestSteps.Add(delayType);
    plan.ChildTestSteps.Add(repeatType);
    plan = await session.SetTestPlanAsync(plan);

    Guid stepA = plan.ChildTestSteps[0].Id;
    Guid stepB = plan.ChildTestSteps[1].Id;
    Guid stepC = plan.ChildTestSteps[2].Id;

    Console.WriteLine("Initial order:");
    plan = await session.GetTestPlanAsync(null);
    foreach (var step in plan.ChildTestSteps)
        Console.WriteLine($"  {step.TypeName}  {step.Id}");

    // --- Reorder: move A and B after C → [C, A, B] ---
    await session.MoveStepsAsync(new HashSet<Guid> { stepA, stepB }, stepC);
    plan = await session.GetTestPlanAsync(null);
    Console.WriteLine("\nAfter MoveSteps (A,B after C):");
    foreach (var step in plan.ChildTestSteps)
        Console.WriteLine($"  {step.TypeName}  {step.Id}");

    // --- Nest: move A and B as children of C ---
    await session.MoveStepsAsChildrenAsync(new HashSet<Guid> { stepA, stepB }, stepC);
    plan = await session.GetTestPlanAsync(null);
    Console.WriteLine("\nAfter MoveStepsAsChildren (A,B into C):");
    Console.WriteLine($"  Root steps: {plan.ChildTestSteps.Count}");
    Console.WriteLine($"  Children of C: {plan.ChildTestSteps[0].ChildTestSteps.Count}");

    // --- Delete step B ---
    await session.DeleteStepsAsync(new HashSet<Guid> { stepB });
    plan = await session.GetTestPlanAsync(null);
    Console.WriteLine("\nAfter DeleteSteps (B):");
    Console.WriteLine($"  Root steps: {plan.ChildTestSteps.Count}");
    Console.WriteLine($"  Children of C: {plan.ChildTestSteps[0].ChildTestSteps.Count}");
}
finally
{
    await session.ShutdownAsync();
    await runner.ShutdownSessionAsync(session.Id);
}

Three operations are available for reorganising steps in a test plan:

• MoveStepsAsync(stepIds, targetStepId) — Moves the specified steps so they appear immediately after the target step at the same nesting level. This is a reorder operation.
• MoveStepsAsChildrenAsync(stepIds, targetStepId) — Moves the specified steps so they become children of the target step. The target must support child steps (e.g. RepeatStep, SequenceStep).
• DeleteStepsAsync(stepIds) — Removes the specified steps from the plan entirely.

All three accept a HashSet<Guid> of step IDs. Moving a step to itself throws an ArgumentException, and moving a parent into one of its own descendants throws an InvalidOperationException.

Querying and searching session logs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NATS.Client.Core;
using OpenTap.Runner.Client;

await using INatsConnection connection = new NatsConnection(new NatsOpts { Url = RunnerClient.DefaultUrl });
await connection.ConnectAsync();
RunnerClient runner = new RunnerClient(connection);
SessionClient session = await runner.StartSessionAsync();

try
{
    // Run a short plan to generate some log entries
    List<TestStepType> types = await session.GetStepTypesAsync();
    TestPlan plan = await session.GetTestPlanAsync(null);
    plan.ChildTestSteps.Clear();
    plan.ChildTestSteps.Add(types.First(step => step.TypeName.Contains("Delay")));
    plan = await session.SetTestPlanAsync(plan);

    Settings s = await session.GetSettingsAsync(plan.ChildTestSteps[0].Id);
    if (s.FirstOrDefault(x => x.PropertyName == "DelaySecs") is TextBoxControl tb)
        tb.StringValue = "1 s";
    await session.SetSettingsAsync(plan.ChildTestSteps[0].Id, s);

    RunStatus status = await session.RunTestPlanAsync(new List<Parameter>());
    while (status.SessionState != SessionState.Idle)
    {
        await Task.Delay(500);
        status = await session.GetStatusAsync();
    }

    // --- Discover available log levels ---
    Dictionary<int, string> levels = await session.LogLevelsAsync();
    Console.WriteLine("Log levels:");
    foreach (var kv in levels)
        Console.WriteLine($"  {kv.Key}: {kv.Value}");

    // --- Discover log sources ---
    List<string> sources = await session.SessionLogSourcesAsync();
    Console.WriteLine($"\nLog sources: {string.Join(", ", sources)}");

    // --- Get per-level log counts ---
    Dictionary<int, int> counts = await session.SessionLogCountsAsync();
    Console.WriteLine("\nLog counts by level:");
    foreach (var kv in counts)
        Console.WriteLine($"  {(levels.TryGetValue(kv.Key, out string? name) ? name : kv.Key.ToString())}: {kv.Value}");

    // --- Fetch paginated log entries ---
    List<int> allLevels = levels.Keys.ToList();
    LogList? logs = await session.GetSessionLogsAsync(
        level: allLevels,
        excludedSource: new List<string>(),
        filterText: "",
        offset: 0,
        limit: 20
    );

    if (logs != null)
    {
        Console.WriteLine($"\nFirst page: {logs.Logs.Count} entries (offset={logs.Offset}, filtered={logs.FilteredCount})");
        foreach (LogEntry entry in logs.Logs.Take(5))
            Console.WriteLine($"  [{entry.Source}] {entry.Message}");
    }

    // --- Search logs for a keyword ---
    List<int> matchIndexes = await session.SessionLogSearchAsync(
        level: allLevels,
        excludedSource: new List<string>(),
        filterText: "",
        searchText: "Delay"
    );
    Console.WriteLine($"\nSearch hits for 'Delay': {matchIndexes.Count} matching entries");
}
finally
{
    await session.ShutdownAsync();
    await runner.ShutdownSessionAsync(session.Id);
}

Session logs can be queried historically using several complementary APIs:

• LogLevelsAsync — Returns a Dictionary<int, string> mapping numeric level IDs to names (e.g. 10 → Debug, 20 → Information, 30 → Warning, 40 → Error).
• SessionLogSourcesAsync — Returns the list of source names that have emitted log entries in the current session.
• SessionLogCountsAsync — Returns per-level entry counts as Dictionary<int, int>.
• GetSessionLogsAsync — Fetches a page of log entries filtered by level, excluded sources, and filter text. The returned LogList includes the entries, the current Offset, FilteredCount, and TotalCount per level.
• SessionLogSearchAsync — Searches log entries matching the filters and returns a list of matching entry indexes. These indexes correspond to positions in the filtered log sequence.

The LogEntry model contains Source, Message, Level (numeric), Timestamp (Unix epoch ticks), and DurationNS.

Future code examples:

• Load from and save to Repository

API Reference

All Runner and Session communication uses NATS request-reply messaging. This document defines the complete set of endpoints, their request/response types, and the data transfer objects (DTOs) used on the wire.

Wire Format

All messages are JSON-serialized. Understanding how different types are encoded on the wire is essential for correct usage:

• Object types (e.g., Image, SetSettingsRequest) are sent as JSON objects: {"Name": "...", "Packages": [...]}
• String values are sent as JSON string literals. For example, SetTestPlanXML expects a JSON-encoded string containing the XML content. The XML must be wrapped in double quotes with proper JSON escaping:

  "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<TestPlan type=\"OpenTap.TestPlan\">..."
  

• Guid values are sent as JSON string literals in standard format: "a8a9baa6-d3b0-446d-a3e2-07bbf9310a71"
• NoInput is sent as an empty JSON object: {}
• NoResponse is returned as an empty JSON object: {}
• Errors from the server are returned as NATS error responses with an OpenTapNatsError header. Fatal errors (e.g., invalid input, server exceptions) are communicated this way rather than in the response body.

Runner APIs

The {base} is in the form of: OpenTap.Runner.{RunnerId}.Request

Session APIs

The {base} is in the form of: OpenTap.Runner.{RunnerId}.Session.{SessionId}.Request

Endpoint Details

SetTestPlanXML

The String request for SetTestPlanXML must contain valid OpenTAP test plan XML (.TapPlan format). The XML content is JSON-encoded as a string literal on the wire (see Wire Format).

The List of String response contains non-fatal deserialization warnings from the OpenTAP serializer. For example, a missing plugin type or a package version mismatch will appear as a string in this list. An empty list means the test plan loaded without issues.

Fatal errors (e.g., invalid XML, corrupt data, or a test plan already executing) are returned as NATS error responses, not in the response body.

Example using the NATS CLI:

nats req OpenTap.Runner.{RunnerId}.Session.{SessionId}.Request.SetTestPlanXML '"<?xml version=\"1.0\" encoding=\"utf-8\"?><TestPlan type=\"OpenTap.TestPlan\"><Steps /></TestPlan>"'

Data Types

NoInput

Empty JSON object {}.

For example, the Session endpoint to abort a test plan takes a NoInput:

nats req OpenTap.Runner.{RunnerId}.Session.{SessionId}.Request.AbortTestPlan "{}"

NoResponse

Empty JSON object {}.

For example, the Session endpoint to abort a test plan returns a NoResponse:

nats req OpenTap.Runner.{RunnerId}.Session.{SessionId}.Request.AbortTestPlan "{}" returns {}.

RunnerStatus

RemoteConnection

StreamInfoStatus

Image

Session

SessionState

Enum with the following values:

• Idle
• Loading
• Executing
• Breaking
• Aborting
• WaitingForUserInput

RunStatus

EditStatus

ErrorResponse

Returned by endpoints such as LoadComponentSettingsFromRepository. Represents a structured error with optional nested inner errors.

NewSessionRequest

NewSessionResponse

DefaultSessionRequest

RegisterSessionRequest

RegisterSessionResponse

SaveDefaultSettings

Note: Repository is accepted by the client library for forward-compatibility but is currently ignored by the server.

RunnerUpdateRequest

RunnerUpdateResponse

GetSettingsRequest

GetTestPlanReferenceResponse

SaveTestPlanToRepositoryResponse

MoveRequest

DeleteRequest

Parameter

EnableMetricsPollingRequest

EnableMetricsPollingResponse

AssetDiscoveryResponse

AssetDiscoveryResult

DiscoveredAsset

SetComponentSettingsRequest

ComponentSettingsBase

ComponentSettingsIdentifier

Extends ComponentSettingsBase. Returned by GetComponentSettingsOverview to identify available component settings without loading their full configuration.

GetComponentSettingsRequest

GetComponentSettingsListItemRequest

ComponentSettingsListItem

SetComponentSettingsListItemRequest

GetComponentSettingDataGridRequest

DataGridControl

SetComponentSettingDataGridRequest

AddComponentSettingDataGridItemTypeRequest

UploadFileRequest

DownloadTapSettingsRequest

RepositoryPackageReference

RepositorySettingsPackageDefinition

AddComponentSettingsListItemRequest

CopyRequest

PasteRequest

PackageSpecifier

Settings

A JSON array of Setting objects. Serialized as [{...}, {...}, ...].

VisualStatus

UnitAttribute

DisplayAttribute

MetaData

ExternalParameter

Layout

ListItemType

Represents a selectable type in a component settings list or data grid.

ColumnDisplayName

Icon

Setting

RepositoryPackageDefinition

TestPlan

SetSettingsRequest

TestPlanValidationErrors

A JSON array of TestStepValidationError objects.

CommonSettings

CommonStepSettingsContext

CommonContext

Interaction

PropertyReferenceRequest

ProfileGroup

Represents a group of component settings profiles.

SetContextMenuRequest

SetDataGridRequest

AddDataGridItemTypeRequest

BreakPoints

GetSessionLogsRequest

LogList

GetSessionSearchRequest

WatchDog

TestStep

TestStepType

Extends TestStep. Returned by GetStepTypes to describe an available test step type and its allowed children.

TestStepValidationError

LogEntry

ValidationError

Verdict

Outcome of a test plan run. Serialized as a PascalCase string.

SlimSession

Compact session summary included in runner heartbeat events.

LayoutMode

Flags enum controlling UI layout of a setting. Values can be combined (bitwise OR).

TypeCode

Maps to System.TypeCode. Indicates the primitive type of a Parameter value. Serialized as a PascalCase string.

Implementation Guide

This document describes the wire-level protocol for building an OpenTAP Runner client in any programming language. It covers NATS connection setup, the custom request-reply pattern, chunked transfer, event subscriptions, session lifecycle, JetStream usage, and JSON serialization rules. Read it alongside the API Reference, which lists every endpoint and DTO.

A client built from these two documents alone should be able to connect to a running Runner, create a session, load and execute a test plan, collect results, and shut down cleanly.

1. NATS Connection Setup

Default URL

A locally installed Runner listens on:

nats://127.0.0.1:20111

When connecting to a cloud-hosted Runner, the URL is supplied by the cloud platform alongside JWT credentials (see Authentication below).

Recommended Connection Options

Match these options on your NATS client to align with the server's expectations:

Authentication

Local Runner — No credentials are required. Connect without authentication.

Cloud Runner — Authenticate with a JWT and NKey seed pair:

• On connection, supply the JWT in the NATS user_jwt field.
• Sign server nonces using the NKey seed (NaCl Ed25519 signature).
• The specific JWT and seed are provided by the KS8500 cloud platform.

Deriving the RunnerId

The RunnerId is the NATS server's reported name, available immediately after the connection handshake as connection.ServerInfo.Name (or the equivalent field in your NATS client library). All subsequent subject construction requires this value.

C# example:

await using INatsConnection connection = new NatsConnection(new NatsOpts
{
    Url = "nats://127.0.0.1:20111"
});

// ServerInfo is populated after the connection is established.
string runnerId = connection.ServerInfo.Name;
string runnerBase = $"OpenTap.Runner.{runnerId}";

2. Subject Hierarchy

All communication uses NATS subjects derived from two base templates. Substitute {RunnerId} with the value derived in section 1.

Base Templates

{SessionId} is a UUID (lowercase, no braces) returned by session-creation endpoints.

Runner Subjects

Session Subjects

{TableName} has NATS-unsafe characters (spaces, dots, wildcards, >) replaced with underscores by the server.

NATS Subscription Wildcards

When subscribing to result streams, use NATS wildcards:

Typical result subscriptions:

{SessionBase}.PlanRun.*                          → plan run start and complete
{SessionBase}.PlanRun.*.StepRun.*                → step run start and complete
{SessionBase}.PlanRun.*.StepRun.*.Result.>       → all result tables
{SessionBase}.PlanRun.*.Logs                     → plan run log batches

3. Request-Reply Protocol

Overview

The Runner does not use the standard NATS request-reply pattern. Instead, the client implements a custom pattern using an explicit reply subscription:

1. Client creates a unique reply subject: {requestSubject}.{NewGuid()}
2. Client subscribes to that reply subject
3. Client publishes the request to {requestSubject} with replyTo set to the reply subject
4. Server sends the response (potentially in multiple chunks) to the reply subject
5. Client reads from the reply subscription until the full response is assembled
6. Client unsubscribes from the reply subject

This is necessary because the response may be chunked across multiple messages (see section 4), which standard request-reply does not support.

Headers

The following NATS message headers are used in requests:

Default Timeout

The default request timeout is 1 minute. There is no built-in retry. If the server does not respond within the timeout, treat the request as failed.

Error Detection

Before deserializing a response, inspect its headers:

• If the OpenTapNatsError header is present (any value), the response body is an error object:

  { "Message": "Human-readable error description" }
  

  Raise an exception with that message.

• If the NATS layer reports no subscribers (NatsNoRespondersException or equivalent), the runner or session is not available on that subject.

Pseudocode

function request(subject, payload, token=null, timeout=60s):
    replySubject = subject + "." + newGuid()
    subscription = nats.subscribe(replySubject)
    try:
        headers = {}
        if token != null:
            headers["Authorization"] = token
        encodedPayload = jsonEncode(payload)   // null → empty byte array
        if len(encodedPayload) > chunkSize:
            publishInChunks(subject, encodedPayload, headers, replySubject)
        else:
            nats.publish(subject, encodedPayload, headers, replyTo=replySubject)

        chunks = []
        deadline = now() + timeout
        while now() < deadline:
            msg = subscription.readNext(deadline - now())
            if msg.hasNoResponders:
                raise NoRespondersError
            if "OpenTapNatsError" in msg.headers:
                raise RunnerError(jsonDecode(msg.body).Message)
            if msg.body.length > 0:
                chunks.append(msg.body)
            if "ChunkSize" not in msg.headers:
                break
            chunkSize = int(msg.headers["ChunkSize"])
            if msg.body.length != chunkSize:
                break   // last chunk

        raise TimeoutError if deadline passed
        return jsonDecode(concatenate(chunks))
    finally:
        subscription.unsubscribe()

C# Example

// BaseClient.RequestAsync<T> — simplified excerpt
INatsSub<byte[]> sub = await connection.SubscribeCoreAsync<byte[]>($"{subject}.{Guid.NewGuid()}");
try
{
    var headers = new NatsHeaders();
    if (useToken && Token != null)
        headers.Add("Authorization", Token);

    byte[] payload = JsonSerializer.SerializeToUtf8Bytes(data, RunnerSerialization.DefaultSerializerOptions);
    if (payload.Length > _chunkSize)
        await PublishInChunksAsync(subject, payload, headers, sub.Subject, ct);
    else
        await connection.PublishAsync(subject, payload, headers, replyTo: sub.Subject, cancellationToken: ct);

    List<byte[]> chunks = new();
    using var cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
    cts.CancelAfter(timeout);

    while (!cts.Token.IsCancellationRequested)
    {
        NatsMsg<byte[]> msg = await sub.Msgs.ReadAsync(cts.Token);

        if (msg.HasNoResponders)
            throw new NatsNoRespondersException();
        if (msg.Headers?.ContainsKey("OpenTapNatsError") == true)
            throw new RunnerException(JsonSerializer.Deserialize<RunnerError>(msg.Data).Message);
        if (msg.Data?.Length > 0)
            chunks.Add(msg.Data);
        if (msg.Headers == null || !msg.Headers.TryGetValue("ChunkSize", out var cs))
            break;
        if (msg.Data?.Length != int.Parse(cs))
            break;
    }

    return JsonSerializer.Deserialize<T>(CombineChunks(chunks));
}
finally
{
    await sub.UnsubscribeAsync();
}

4. Chunked Transfer Protocol

Large messages — in both directions — are split into chunks. Every endpoint that accepts or returns a payload may involve chunking. Clients must handle chunks even for small payloads, because the server may choose to chunk any response.

Chunk Size

MaxPayload is advertised by the NATS server in the connect INFO message (connection.ServerInfo.MaxPayload).

Sending Chunks (Client → Server)

1. Encode the full payload as UTF-8 JSON bytes.
2. If the payload fits within one chunk, send it as a single message with no chunk headers.
3. If the payload exceeds one chunk:
   • Add headers to every message:
     • RequestId: a new UUID string (same value for all chunks of one request)
     • ChunkSize: the chunk size in bytes (integer string)
     • ChunkNumber: 1-indexed position of this chunk (integer string)
   • Send chunks sequentially.
   • Trailing empty message: if the final chunk's data length is exactly equal to ChunkSize (i.e., no partial last chunk), send one additional message with an empty body. This is also required when the total payload is empty (zero bytes). This sentinel tells the receiver that no more chunks are coming.

Receiving Chunks (Client ← Server)

Read messages from the reply subscription in a loop:

chunks = []
while true:
    msg = read_next()
    if msg.body.length > 0:
        chunks.append(msg.body)
    if "ChunkSize" not in msg.headers:
        break   // single-message response (no chunking in use)
    chunkSize = int(msg.headers["ChunkSize"])
    if msg.body.length != chunkSize:
        break   // this message is shorter than a full chunk → it is the last one
// reassemble: concatenate all chunks in order
payload = concatenate(chunks)

The last chunk is detected by its size being strictly less than ChunkSize (or empty). The server never sends a ChunkNumber count in advance; the client discovers the end by size.

Server-Side Reassembly

The server reassembles chunked requests using a RequestHandler keyed on RequestId:

• Supports out-of-order delivery (chunks are stored by index).
• Maximum 1000 chunks per request (~1 GB at the default NATS max payload).
• Incomplete requests expire after 60 seconds of inactivity and are discarded.

C# Example (Sending)

// PublishInChunksAsync — from BaseClient.cs
private async Task PublishInChunksAsync(string subject, ReadOnlyMemory<byte> payload,
    NatsHeaders headers, string replyTo, CancellationToken ct)
{
    headers.Add("RequestId", Guid.NewGuid().ToString());
    headers.Add("ChunkSize", _chunkSize.ToString());

    int sent = 0, chunkNumber = 1;
    while (sent < payload.Length)
    {
        var chunkHeaders = CloneHeaders(headers);
        chunkHeaders["ChunkNumber"] = chunkNumber.ToString();
        int count = Math.Min(payload.Length - sent, _chunkSize);
        await connection.PublishAsync(subject, payload.Slice(sent, count).ToArray(),
            chunkHeaders, replyTo, cancellationToken: ct);
        sent += count;
        chunkNumber++;
    }

    // Sentinel: send empty message if last chunk was exactly full (or payload was empty)
    if (payload.Length % _chunkSize == 0)
    {
        headers["ChunkNumber"] = chunkNumber.ToString();
        await connection.PublishAsync(subject, Array.Empty<byte>(), headers, replyTo, cancellationToken: ct);
    }
}

5. Session Lifecycle

Session States

Sessions progress through a defined set of states. The current state is always available via GetStatus and is also broadcast in SessionStateChanged events.

[Created] → Loading → Idle → Executing → Idle
                                ↕
                            Breaking
                                ↕
                        WaitingForUserInput
                                ↓
                           Aborting → Idle

Creating a Session

Call one of the session-creation endpoints on the Runner:

All return a Session object containing the session's UUID (Id). Construct the session base subject immediately:

sessionBase = "OpenTap.Runner.{RunnerId}.Session.{session.Id}"

Waiting for Readiness

A newly created session starts in the Loading state and is not immediately ready to accept commands. Wait for SessionState.Idle before sending requests.

Option A — Poll:

while true:
    status = request(sessionBase + ".Request.GetStatus")
    if status.SessionState == "Idle":
        break
    sleep(250ms)

Option B — Subscribe:

subscribe(sessionBase + ".Events.SessionStateChanged", handler = function(event):
    if event.RunStatus.SessionState == "Idle":
        markReady()
)

Shutdown Sequence

Always shut down sessions explicitly to release resources:

1. Send Shutdown to the session:

   request(sessionBase + ".Request.Shutdown")
   

2. Send ShutdownSession to the runner (passing the session UUID):

   request(runnerBase + ".Request.ShutdownSession", sessionId)
   

Watchdog / Heartbeat

If your client needs to maintain a long-lived session without activity, monitor the session heartbeat events and ensure the NATS connection stays alive.

6. Event and Subscription Model

Events are published as fire-and-forget core NATS messages (not JetStream). Subscribe before creating a session or starting a test plan to avoid missing early events.

Runner Events

Subscribe to these on the runner base subject (OpenTap.Runner.{RunnerId}.{suffix}):

StartedLifetimeEvent

Published when the Runner process has started.

Subject: Events.Lifetime.Started

No payload fields.

StoppedLifetimeEvent

Published when the Runner process is stopping.

Subject: Events.Lifetime.Stopped

No payload fields.

HeartbeatLifetimeEvent

Published every 15 seconds. Contains a snapshot of all currently active sessions.

Subject: Events.Lifetime.Heartbeat

RunningEvent

Published when the Runner's overall executing state changes.

Subject: Events.Running

MetadataUpdatedEvent

Published when Runner-level metadata changes (e.g. after a session is registered or updated).

Subject: Events.MetadataUpdated

Session Events

All session events are published to {SessionBase}.Events.{EventName}. Each event type has its own subject; there is no single wildcard subject that captures all session events. Subscribe to each individually.

TypeCacheInvalidatedEvent

Published when the session's type cache has been invalidated and must be refreshed.

Subject: Events.TypeCacheInvalidated

No payload fields.

TestPlanSettingsChangedEvent

Published when the test plan settings associated with the session have changed.

Subject: Events.TestPlanSettingsChanged

No payload fields.

SessionTimeoutEvent

Published immediately before the session terminates due to watchdog inactivity timeout.

Subject: Events.SessionTimeout

No payload fields.

StartingEvent

Published when a test plan execution is about to begin (sequencer is transitioning into Executing).

Subject: Events.Starting

No payload fields.

StartedEvent

Published after a test plan execution has started and the first step is about to run.

Subject: Events.Started

No payload fields.

HeartbeatEvent

Published every 5 seconds while the session is alive. Use this to monitor session health and watchdog state.

Subject: Events.Heartbeat

WatchDog fields:

StoppingEvent

Published when a test plan execution is in the process of stopping (e.g. after abort or normal completion, before teardown is complete).

Subject: Events.Stopping

No payload fields.

StoppedEvent

Published when a test plan execution has fully stopped and the session has returned to Idle.

Subject: Events.Stopped

No payload fields.

TestPlanChangedEvent

Published whenever the in-memory test plan is modified (step added, removed, reordered, or a property changed).

Subject: Events.TestPlanChanged

EditStatus fields:

SessionStateChangedEvent

Published whenever the session's execution state changes (e.g. Idle → Executing, Executing → Idle). This is the primary event for tracking execution progress.

Subject: Events.SessionStateChanged

SettingsChangedEvent

Published when a component settings group has been modified.

Subject: Events.SettingsChanged

TestStepChangedEvent

Published when a specific test step has been modified.

Subject: Events.TestStepChanged

BreakEvent

Published when execution pauses at a breakpoint. The session enters Breaking state.

Subject: Events.Break

UserInputRequestedEvent

Published when a test step requires user input before it can continue. The session enters WaitingForUserInput state.

Subject: Events.UserInputRequested

UserInputRequestCompletedEvent

Published when a pending user input request has been resolved (answered, cancelled, or timed out).

Subject: Events.UserInputRequestCompleted