Version5.0.0
System Exclusive
Everything we have looked at so far was decided in 1983 and has not changed since. A Note On is three bytes, a Control Change is three bytes, and every synthesizer in the world agrees on what they mean. That agreement is MIDI's great strength, and it is also a cage. There is no standard message for set the feedback of operator six, because operator six is a Yamaha idea, and there is no standard message for send me your current sound, because every manufacturer stores a sound differently.
The designers of MIDI knew this would happen, and they left a door open. The System Exclusive message, SysEx for short, is a message whose content MIDI does not define at all. Only the frame around it is standard; what goes inside is the manufacturer's business. It is the one message type with no fixed length, and it is the message type the Electra One spends most of its time on.
The frame
A SysEx message starts with the status byte 0xF0 and ends with 0xF7. Everything between them is data, and all of it must be data bytes, with values from 0x00 to 0x7F:
0xF0 <manufacturer ID> <data ...> 0xF7Because the end is marked by 0xF7 rather than by a fixed length, a SysEx message can be three bytes long or several thousand. A single parameter change for a DX7 is seven bytes. A DX7 voice is 163 bytes. A whole bank of thirty-two voices is over four thousand, and a firmware update can be hundreds of thousands.
The rule that data bytes stay below 0x80 is the same one that holds the rest of MIDI together, and it has a consequence here that we will deal with in the next chapter: a value that needs eight bits or more has to be split across several bytes before it can travel inside a SysEx message.
Who it is for
The first data byte is the manufacturer ID, and it exists so that an instrument can tell at a glance whether a SysEx message concerns it. Yamaha is 0x43, Roland is 0x41, Korg is 0x42, Sequential is 0x01. The one-byte numbers ran out long ago, so newer companies have a three-byte ID that starts with 0x00; the Electra One's own is 0x00 0x21 0x45, and you will see it in front of every message the controller exchanges with the editor.
Two IDs are reserved for everyone. 0x7E introduces universal messages that are not real-time, and 0x7F introduces universal real-time messages. The best known of them is the identity request, which asks whatever is listening to say what it is:
0xF0 0x7E 0x7F 0x06 0x01 0xF7The answer names the manufacturer, the model and the firmware version, and it is how an editor can find its instrument on a port without being told.
A SysEx message is not tied to a MIDI channel. Every device on the port receives it and decides for itself whether to answer. Most manufacturers put a device number into the message instead, usually right after the manufacturer ID, so that two identical synthesizers on one cable can still be told apart. By convention 0x7F in that place means all of you.
What goes inside
From here on, every manufacturer does it their own way, and the only reliable guide is the instrument's manual, usually in a chapter called MIDI Implementation or System Exclusive Format near the back. Yet the manuals describe a surprisingly small number of patterns, and once you have seen them you can read most of them.
A parameter change names a parameter and gives it a value. This is a Yamaha DX7 being told to set its algorithm to number 18:
0xF0 0x43 0x10 0x01 0x06 0x11 0xF70x43 is Yamaha, 0x10 carries both the message kind and the channel, 0x01 0x06 is the parameter number 134 split over two bytes, and 0x11 is the value. Seven bytes, and a different parameter number or value is a different setting on the synthesizer.
A request asks the instrument to send something. The DX7's request for its current voice is three data bytes:
0xF0 0x43 0x20 0x00 0xF7A dump is the instrument's answer: a header that says what is coming, followed by the data, often followed by a checksum. The DX7 answers the request above with 163 bytes, and inside them are the 145 parameters of the voice and the ten characters of its name.
The request and dump pair is the pattern that matters most. An editor could guess what an instrument is set to, but it does not have to: it asks, the instrument answers, and the controls on the screen are filled from the answer. Instruments answer when they are asked rather than announcing every change, because a stream of unsolicited dumps would flood the cable.
A checksum is a byte at the end that the receiver can recompute from the data, to check that nothing was lost on the way. Roland's is the best known: add up the bytes of the address and the data, and send whatever makes the sum come out to a multiple of 128. A message that fails the check is thrown away. The Electra One can compute the common checksums for you, in a template or in Lua.
Trying it
Open the Console tab and send the identity request from above, using the console's keyword for SysEx:
syx 7eh 7fh 06h 01hThe F0 and F7 are added for you. If an instrument that answers identity requests is on the port the console sends to, its reply appears in the list as a syx line, and expanding the line shows the manufacturer ID in the reply. Nothing arriving means nothing on that port answers, which is itself a useful thing to know about an instrument.
Then look at the message the Electra One itself sends when a control uses SysEx. The screenshot below shows the console with a DX7 parameter change expanded, which is exactly the seven-byte message from this chapter, sent by a control.
Real-time messages in the middle
A SysEx message may be long, and MIDI clock does not wait. The specification therefore allows a System Real-Time byte to appear anywhere inside a SysEx message, even between two data bytes:
0xF0 0x43 0x10 0x01
0xF8 a clock tick
0x06 0x11 0xF7The receiver handles the clock and continues reading the SysEx message as if nothing had happened. It is the same rule we met with running status, and for the same reason: a single-byte real-time message cannot be confused with anything else. If you ever write a SysEx parser of your own, in Lua or anywhere else, this is the case that will catch you if you forget it.
SysEx on the Electra One
The reason the Electra One can be an editor for almost any instrument is that it treats SysEx as a first-class citizen. A control can send a SysEx message with its value placed into a byte, or into a few bits of a byte. A device can hold one template that every control of the instrument shares, and can listen for the same message coming back so that the instrument moves the controls. A preset can carry a request, recognise the dump that answers it, and take the dump apart into the controls with rules. And where the rules stop, Lua takes over, with the whole dump handed to the script as a block of bytes.
Each of those is a tutorial in the third part of this book, and they build on each other in order:
All four use the DX7 messages from this chapter.
What comes next
Twice in this chapter a number too large for one data byte was split over two: the parameter number 134 became 0x01 0x06, and a 163-byte dump keeps its values in seven-bit pieces. The last chapter of this part is about that splitting, the handful of bit operations behind it, and how to read a manual that describes a byte bit by bit.