Lua tests

A preset can carry two test files beside its scripts: main-tests.lua, which tests main.lua, and router-tests.lua, which tests router.lua. A test file is a Lua file that declares cases. The controller runs the file on demand, inside the preset's own Lua state, with the outgoing MIDI held back and recorded, and answers with a JSON document that says which cases passed and why the others did not. This page describes the files, the test library they use, how a run is started, and what comes back.

Note

Lua tests are available from firmware 5.0.0c. They need the preset under test to be loaded on the controller: a test file is run against the preset on the screen.

The test files

File	Tests	Editor tab
main-tests.lua	main.lua	Preset tests
router-tests.lua	router.lua	Router tests

The files belong to the project. The editor saves them with it, loads them when the project is opened, and keeps them in the account like the scripts. A test file is sent to the controller only when it is run, and the controller keeps nothing: it runs the file, answers, and forgets it. There is no upload, no slot file and no file transfer type for a test file.

Both files use the same test library. main-tests.lua is the place for cases that turn knobs, press pads and receive MIDI; router-tests.lua is the place for cases that push a message through the router. The split is for the reader, not for the firmware: test.route() works from either file.

Asking and approving

A test file is written in two steps, and the second one is done by the person who plays the instrument.

A case is first written unapproved. It says what the script is meant to do, but nobody has yet confirmed on the instrument that this is what the preset should do:

test.case("cutoff sends CC 74 inverted", function()
  test.turn(1, 100)
  local sent = test.sent()
  test.count(sent, 1)
  test.eq(sent[1].controller, 74)
  test.eq(sent[1].value, 27)
end)

The person then tries the behaviour on the controller - turns the cutoff, watches the synth - and says whether it is right. Only then is the case marked approved, with the date of the confirmation:

test.case("cutoff sends CC 74 inverted", { approved = "2026-09-22" }, function()
  ...
end)

An approved case is a contract. It records a behaviour the user has heard and accepted, so a change to the script that turns an approved case red is wrong until the user says otherwise. The rules that follow from that:

• An approval is added only after the user has confirmed the behaviour on the instrument. Passing is not approval; a case that passes has only shown that the script does what the case says, not that the case says the right thing.
• An approval is never added, changed or removed by the person - or the assistant - who wrote the code. If an approved case has to change because the preset is meant to behave differently now, that is said first, and the case is changed and re-approved after the user has agreed.
• Before changing a script that already has approved cases, run the tests. Green before and red after is the change; red before is something else.

The firmware does not interpret the date. It reports it back in the result so that the editor can show which cases are approved and the counters can say how many.

Writing a test file

The file runs in the preset's Lua state, after main.lua has run. Every global the script defined is in reach: a formatter can be called directly, a variable the script keeps can be read, parameterMap.get() says what the map holds. The test library is global there, like midi and controls.

The top level of the file runs once and registers cases with test.case(). Nothing the top level does is recorded, and it is not a case: a helper function or a shared table belongs there.

local function lastSent()
  local sent = test.sent()
  return sent[#sent]
end

test.case("the pad mutes", function()
  test.press(2)
  test.eq(lastSent().controller, 20)
  test.eq(lastSent().value, 127)
end)

Cases run in file order. Each starts with nothing recorded, and each is run with the preset as the previous cases left it: a value turned in one case is still turned in the next. A case that needs a known starting point sets it itself.

Assertions

An assertion that does not hold ends the case with the status failed and a message that says what was expected and what was found. Every assertion takes an optional last argument, why, which is appended to the message after --.

Assertion	Fails when	Message
test.eq(actual, expected [, why])	actual ~= expected	eq: expected 27, got 100
test.ne(actual, other [, why])	actual == other	ne: both are 3
test.ok(condition [, why])	condition is false or nil	ok: expected a true value, got nil
test.count(list, n [, why])	#list ~= n	count: expected 1, got 2
test.none(list [, why])	#list ~= 0	none: expected nothing, got 1
test.fail(why)	always	fail -- why

Strings are quoted in the message, so test.eq("a", "b") reports eq: expected "b", got "a". test.count() and test.none() count anything that is not a table as empty.

Anything else that raises inside a case - a nil index, a firmware function given a bad argument, error() - ends the case with the status error and the Lua error message. The two are counted apart in the result, so a case that found a wrong value and a case that blew up are told apart.

Driving the preset

Four functions do what a knob, a pad, a wire and the timer do. Each returns after everything it set in motion has been processed, so the next line can assert on the outcome.

test.turn(1, 100)                      -- control 1 turned to 100
test.press(2)                          -- pad 2 pressed: on when off, off when on
test.receive(PORT_1, { type = CONTROL_CHANGE, channel = 1,
                       controller = 74, value = 33 })
test.tick(3)                           -- timer.onTick() three times

test.turn() takes the value as the display shows it - the number a formatter is given - and clamps it to the value's min and max. It goes through the parameter map the way a knob does, so everything a knob sets off runs, in the firmware's own order:

1. the control's own message is sent, and recorded;
2. parameterMap.onChange() runs, if the script defines one;
3. the value's function runs;
4. the value's formatter runs.

Whatever the function sends is recorded after the control's own message. A value with a virtual message sends nothing of its own, so only what the function sends is recorded.

test.press() toggles a pad between its message's onValue and offValue. On a control that is not a pad it sets the onValue.

test.receive() delivers a message as if it had arrived on the port: the preset's devices resolve it, the parameter map takes it, and the midi.on* callbacks run. The interface is MIDI_IO unless a third argument says otherwise. A value that arrives from the instrument is not sent back, so test.none(test.sent()) after a receive holds. A received message does not pass through router.lua; use test.route() for that.

test.tick(n) fires timer.onTick() n times - once when n is left out - and only while the timer is enabled. It returns how many ticks fired, so a case can check that a disabled timer fires nothing.

Testing the router

local passed, sent = test.route("midiIo1",
  { type = NOTE_ON, channel = 5, note = 60, velocity = 100 })

test.route() runs the preset's router.lua on one message arriving on the named port. passed is the router's verdict: true when the routing matrix would carry on with the message, false when the script owned it. sent is the list of messages the script sent while handling it - with m:to() or a port's send function - in order.

The router is loaded fresh from the preset's router.lua for the first test.route() of each case, so init() has run and the parameters are what init() declares; it is unloaded when the case ends. A preset with no router.lua raises test.route: this preset has no router.lua.

The port is one of "midiIo1", "midiIo2", "usbDev1", "usbDev2", "ctrl", "usbHost1" and "usbHost2". A message cannot arrive on "all", "preset" or "midiControl".

The router's sends are recorded and not sent, like everything else in a case, and they also appear in test.sent().

What is recorded

While a case runs, everything the application thread sends is held and recorded instead of going to a port, and every entry the firmware makes into the script is noted. Three functions read the record.

test.sent() returns the messages sent since the case began, or since the last test.clear(). A message sent by the preset - by a control, or by midi.send*() - looks like this:

{ kind = "sent", port = 0, interface = USB_DEV, name = "usbDev1",
  type = CONTROL_CHANGE, channel = 1, data1 = 74, data2 = 27,
  controller = 74, value = 27 }

• port is the port number, interface the interface constant. A send to ALL_INTERFACES has interface = ALL_INTERFACES.
• name is the router's name for that interface and port - "midiIo1", "usbDev2", "all" - and is left out when the pair has no name.
• type is the message type constant, NOTE_ON, CONTROL_CHANGE and the rest, and channel is present on channel messages only.
• data1 and data2 are the bare bytes; the named fields the type has come beside them, as in the table under Message tables.
• a SysEx message is { type = SYSEX, length = n, data = { 0xF0, ..., 0xF7 } }.

A message the router sent is addressed the way router.lua thinks, by port name: port is the destination's name - "usbDev1", "preset", "midiControl" - and portNumber the cable.

test.calls() returns the calls the firmware made into the script, in order:

{ kind = "call", role = "function", name = "onCutoff" }

role is one of "function" (a value's function), "formatter", "onChange" (parameterMap.onChange), "onEvent" (a control event callback), "onTick", "onMidi" (any midi.on* callback), "parameterFunction" and "templateFunction" (the Lua functions a SysEx template names). Any other hook carries the firmware's own name for it. name is the function's name where the firmware knows it.

test.timeline() returns both kinds merged, in the order they happened. Each entry has kind, "sent" or "call". It is the way to check that one thing happened before another: that a function sent its message after the control's own, for example.

test.clear() forgets what has been recorded so far in the case. It is what a case calls after setting up, so that the assertions see only the step under test.

Message tables

test.receive() and test.route() take a message as a table, written the way midi.sendMessage() takes one: a type and the fields that type has. channel is 1 when left out. A type with no named fields, or a message you would rather spell in bytes, takes data1 and data2.

type	Fields
NOTE_ON, NOTE_OFF	channel, note, velocity
POLY_PRESSURE	channel, note, pressure
CONTROL_CHANGE	channel, controller, value
PROGRAM_CHANGE	channel, program
CHANNEL_PRESSURE	channel, pressure
PITCH_BEND	channel, value (0 to 16383)
SONG_POSITION	position
SONG_SELECT	song
CLOCK, START, CONTINUE, STOP, and the other system messages	none
any	data1, data2

The same names come back on a recorded entry, so an assertion reads the way the message was written. test.receive() cannot deliver a SysEx message.

Running the tests

From the editor

The Lua tab of the editor has four tabs at the top: Main, Router, Preset tests and Router tests. On a tests tab the log is replaced by a results panel, and a Run button sends the tab's file to the connected controller. The panel shows a summary line and one row per case: the name, a mark for passed, failed or error, approved and the date where the case carries one, and the message under a case that did not pass. A file that did not compile shows its error as the one row.

The preset the tests are written for has to be on the controller: send it first, then run.

From the command line

electraone lua test --file main-tests.lua
electraone --pretty lua test --file router-tests.lua
cat main-tests.lua | electraone lua test --file -

The command prints the result document. Its exit code is 0 when the run is clean - no top-level error, failed and errors both zero - 1 otherwise, and 2 when the controller did not answer in time. A run may take seconds, because every case runs on the controller's application thread, so the command waits 30 s rather than the usual 3 unless --timeout says otherwise.

Over SysEx

A test file is sent as a query, and the result is the reply. The reply comes on the USB device port the request came in on, after the last case has run; no ACK follows it. Hosts should wait at least 30 seconds.

Request

0xF0 0x00 0x21 0x45 0x02 0x42 test-file-source 0xF7

Response

0xF0 0x00 0x21 0x45 0x01 0x42 result-json-data 0xF7

The result document

{
  "cases": 3, "passed": 2, "failed": 1, "errors": 0, "approved": 1,
  "firmware": "5.0.0c", "model": "mk2",
  "results": [
    { "name": "cutoff sends CC 74 inverted", "approved": "2026-09-22",
      "status": "passed" },
    { "name": "unmute sends the current cutoff once",
      "status": "failed",
      "message": "count: expected 2, got 3 -- one catch-up message, not a burst" },
    { "name": "the pad mutes",
      "status": "passed" }
  ]
}

Field
cases	how many cases the file declared
passed, failed, errors	how many ended each way; failed is an assertion, errors a case that raised
approved	how many cases carry an approval
firmware, model	the firmware version and the model that ran the file: "mk2", "mini" or "mk3"
results	one entry per case, in file order
results[].status	"passed", "failed" or "error"
results[].message	the assertion's message or the Lua error; present when the case did not pass, cut to fit if long
results[].approved	the case's approval date, present only when the case declared one

A file that cannot run answers the same document with a top-level error and every counter at zero:

{ "error": "main-tests.lua:3: '=' expected near 'foo'",
  "cases": 0, "passed": 0, "failed": 0, "errors": 0, "approved": 0,
  "firmware": "5.0.0c", "model": "mk2" }

That is the answer for a file that does not compile, a file whose top level raises, and a request the controller cannot serve: no preset script is loaded, the script is stopped in the debugger, or the test library is not available in this preset. A host checks error before it reads the counters.

After a run

The cases leave the preset as they turned it: values moved, the timer perhaps enabled, variables changed. After a run that produced results the controller reloads the preset in the background - the same reload a script's preset:reload() schedules - so the values and the script's state go back to what the user had, and the screen is redrawn. A run that answered a top-level error does not reload.

Limits

• The file is carried in a SysEx message, so it is ASCII: a byte with its top bit set ends the message.
• The tests run against the preset on the screen. They cannot name another slot.
• test.receive() runs the preset's own handling and the midi.on* callbacks. It does not go through router.lua; test.route() does, and nothing else.
• What a case sends is held and recorded, and not sent to any port. SysEx to the CTRL port is the exception: the logger and the SysEx API answer from there, and they keep working during a run. print() from a case reaches the log as usual.
• Every case runs on the application thread while the run holds it. Knobs and the screen wait for the run to finish; a file with many slow cases keeps the controller busy for that long.
• A failure message is cut at about 240 characters.

A worked example

A small preset for a soft synth on the computer:

• Device 1, Synth, on USB device port 1, channel 1.
• Control 1, CUTOFF, a fader with a virtual message, parameter 74, range 0 to 127. Its function is onCutoff and its formatter formatCutoff. The synth wants CC 74 the other way up, so the control sends nothing itself and the function sends the inverted value.
• Control 2, MUTE, a pad on CC 20 with onValue 127 and offValue 0. Its function is onMute. Unmuting sends the cutoff again, once, because the synth forgets it while muted.
• A router that forwards notes from the DIN input to the synth on channel 1.

main.lua:

local function sendCutoff(value)
  midi.sendControlChange(USB_DEV, PORT_1, 1, 74, 127 - value)
end

function onCutoff(valueObject, value)
  sendCutoff(value)
end

function formatCutoff(valueObject, value)
  if value == 0 then
    return "closed"
  elseif value == 127 then
    return "open"
  end
  return tostring(value)
end

function onMute(valueObject, value)
  if value == 0 then
    sendCutoff(parameterMap.get(1, PT_VIRTUAL, 74))
  end
end

router.lua:

function init()
  router.watch{ types = { NOTE_ON, NOTE_OFF }, from = { ports.midiIo1 } }
end

function onMidi(m)
  m.channel = 1
  m:to(ports.usbDev1)
  return false
end

main-tests.lua. The first two cases have been tried on the synth and approved; the third is still waiting for the user to confirm that one catch-up message is what they want:

test.case("cutoff sends CC 74 inverted", { approved = "2026-09-22" }, function()
  test.turn(1, 100)

  local sent = test.sent()
  test.count(sent, 1, "the virtual message sends nothing of its own")
  test.eq(sent[1].name, "usbDev1")
  test.eq(sent[1].type, CONTROL_CHANGE)
  test.eq(sent[1].channel, 1)
  test.eq(sent[1].controller, 74)
  test.eq(sent[1].value, 27)

  local calls = test.calls()
  test.eq(calls[1].role, "function")
  test.eq(calls[1].name, "onCutoff")
  test.eq(calls[2].role, "formatter")
end)

test.case("the pad mutes and unmutes", { approved = "2026-09-22" }, function()
  test.turn(2, 0)                      -- start from off, whatever came before
  test.clear()

  test.press(2)
  test.press(2)

  local sent = test.sent()
  test.eq(sent[1].controller, 20)
  test.eq(sent[1].value, 127)
  test.eq(sent[2].controller, 20)
  test.eq(sent[2].value, 0)
end)

test.case("unmute sends the current cutoff once", function()
  test.turn(1, 100)
  test.turn(2, 0)
  test.press(2)                        -- mute
  test.clear()

  test.press(2)                        -- unmute

  local sent = test.sent()
  test.count(sent, 2, "one catch-up message, not a burst")
  test.eq(sent[1].controller, 20)
  test.eq(sent[1].value, 0)
  test.eq(sent[2].controller, 74)
  test.eq(sent[2].value, 27)
end)

test.case("the formatter names the ends of the range", function()
  test.eq(formatCutoff(nil, 0), "closed")
  test.eq(formatCutoff(nil, 127), "open")
  test.eq(formatCutoff(nil, 64), "64")
end)

test.case("a value from the synth moves the pad and is not echoed", function()
  test.receive(PORT_1, { type = CONTROL_CHANGE, channel = 1,
                         controller = 20, value = 127 }, USB_DEV)
  test.eq(parameterMap.get(1, PT_CC7, 20), 127)
  test.none(test.sent())
end)

router-tests.lua:

test.case("notes from the keyboard reach the synth on channel 1",
          { approved = "2026-09-22" }, function()
  local passed, sent = test.route("midiIo1",
    { type = NOTE_ON, channel = 5, note = 60, velocity = 100 })

  test.eq(passed, false, "the router owns the note")
  test.count(sent, 1)
  test.eq(sent[1].port, "usbDev1")
  test.eq(sent[1].type, NOTE_ON)
  test.eq(sent[1].channel, 1)
  test.eq(sent[1].note, 60)
  test.eq(sent[1].velocity, 100)
end)

test.case("clock is left to the matrix", function()
  local passed, sent = test.route("midiIo1", { type = CLOCK })
  test.eq(passed, true)
  test.none(sent)
end)

Running the first file answers cases: 5, passed: 5, approved: 2 on a preset that behaves as described. Changing onMute() to send the cutoff on every press turns the third case red with count: expected 2, got 3 -- one catch-up message, not a burst - which is the case doing its job, and, once it is approved, the change being wrong.

Lua tests API reference

Cases

test.case(name [, options], fn)

Registers a case. Cases run in the order they were registered, after the whole file has run. A name that is not a string, or a case that is not a function, raises at registration, which the result reports as a top-level `error`.

Parameters

name

string, the name of the case. It is reported back as written.

options

table, optional. { approved = "YYYY-MM-DD" } marks the case approved on that date. Any other field is ignored.

fn

function, the case. It takes no arguments.

Assertions

test.eq(actual, expected [, why])

Fails the case unless `actual == expected`. The message is `eq: expected 27, got 100`; strings are quoted.

Parameters

actual

any value.

expected

any value.

why

string, optional. Appended to the failure message.

test.ne(actual, other [, why])

Fails the case when `actual == other`. The message is `ne: both are 3`.

Parameters

actual

any value.

other

any value.

why

string, optional.

test.ok(condition [, why])

Fails the case when `condition` is `false` or `nil`. The message is `ok: expected a true value, got nil`.

Parameters

condition

any value.

why

string, optional.

test.count(list, n [, why])

Fails the case unless `#list == n`. The message is `count: expected 1, got 2`.

Parameters

list

table, a list. Anything that is not a table counts as empty.

n

integer, the expected length.

why

string, optional.

test.none(list [, why])

Fails the case unless `list` is empty. The message is `none: expected nothing, got 1`. `test.none(test.sent())` is the way to say that nothing was sent.

Parameters

list

table, a list. Anything that is not a table counts as empty.

why

string, optional.

test.fail(why)

Fails the case. The message is `fail -- ` followed by the reason.

Parameters

why

string, the reason.

Driving the preset

test.turn(controlId, value [, valueId])

Sets the value as a knob would. The control's message is sent and recorded, `parameterMap.onChange()` runs, then the value's function and its formatter, in that order. Returns after all of them have run.

Parameters

controlId

integer, the id of a control in the preset. A control that does not exist raises.

value

integer, the display value, as a formatter is given it. Clamped to the min and max of the value.

valueId

string, optional. The id of the value on a control with several; the first value when left out. A value id the control does not have raises.

test.press(controlId [, valueId])

Presses a pad: the message goes to its `offValue` when it stands at its `onValue`, and to its `onValue` otherwise. Everything a turn sets off runs here too.

Parameters

controlId

integer, the id of a control in the preset.

valueId

string, optional. As for test.turn().

test.receive(port, message [, interface])

Delivers the message as if it had arrived on the port: devices resolve it, the parameter map takes it, the `midi.on*` callbacks run. Returns after the callbacks have run. The message does not go through `router.lua`, and a SysEx message cannot be delivered this way.

Parameters

port

integer, a port constant: PORT_1, PORT_2 or PORT_CTRL.

message

table, a message as in Message tables. A table without a type raises.

interface

integer, optional. An interface constant; MIDI_IO when left out.

test.tick([n])

Fires `timer.onTick()` `n` times, one period each, without waiting for the period. Nothing fires while the timer is disabled.

Parameters

n

integer, optional, how many ticks. 1 when left out. A negative count raises.

Returns

integer, how many ticks fired: n while the timer is enabled, 0 while it is not.

test.route(port, message)

Runs the preset's `router.lua` on one message. The router is loaded from the preset's slot on the first call in a case, `init()` and all, and unloaded when the case ends. A preset without a `router.lua` raises.

Each entry of sent has port set to the destination's name and portNumber to its cable, beside the fields of a recorded message.

Parameters

port

string, the port the message arrives on: "midiIo1", "midiIo2", "usbDev1", "usbDev2", "ctrl", "usbHost1" or "usbHost2". "all", "preset" and "midiControl" raise, as does a name that is not a port.

message

table, a message as in Message tables.

Returns

passed, sent: a boolean, true when the routing matrix would carry on with the message, and a list of the messages the script sent, in order.

Reading the record

test.sent()

The messages the application thread and the router sent during the case. See [What is recorded](#what-is-recorded) for the fields.

Returns

a list of the messages sent since the case began or since test.clear(), oldest first. Each has kind = "sent".

test.calls()

The entries into user Lua during the case: functions, formatters, `parameterMap.onChange`, event callbacks, `timer.onTick`, `midi.on*` callbacks, and the functions a SysEx template names.

Returns

a list of the calls the firmware made into the script, oldest first. Each has kind = "call", role and name.

test.timeline()

`test.sent()` and `test.calls()` merged. `kind` tells the two apart.

Returns

a list of both kinds of entry, in the order they happened.

test.clear()

Forgets everything recorded so far in the case. The next `test.sent()`, `test.calls()` and `test.timeline()` start from here.

See also

• Preset Lua Extension - the script main-tests.lua tests
• Router Lua Extension - the script router-tests.lua tests
• Lua Editor and Debugger - the tabs the files are written on
• SysEx Implementation - the protocol the request travels on