Writing a Server Script
In this tutorial, we will show you how to listen and respond to events generated by the currently opened transport session in order to create a server which responds to incoming commands.
We will also demonstrate a very powerful technique of extracting text-based commands using the Jancy regex switch statement.
Design Considerations
You may think, transport plugins are written in Jancy, and the same language is used in in-app scripting. Receiving events from transport plugins and handling those in your in-app scripts shouldn’t be that hard, right? Well, there is a catch.
Different transports behave completely differently. They generate different kinds of events with different parameters. Establishing connections works differently, as well, requires different sets of parameters, and can be instant for some transports, and lengthy for others. Reading from different transports may have different semantics, too. Bottom line is, devising a universal API which would be applicable to any transport – is nearly impossible. Attempting to do so will yield some sort of the so-called God interface, which is never a good design.
Therefore, IO Ninja is using a different approach to handling transport events in your in-app scripts. Instead of listening for IO events from transport sessions – which is very hard to design a uniform API for, your in-app scripts will be listening for new log records! The IO Ninja Log API is already uniform for all kinds of sessions (difference lies in transport-specific log record codes and record parameters). So in-app scripting is using it as a uniform way to handle events from all kinds of sessions.
Event-handling Entry Point
Your event-handling entry point will be called every time a new record is added to the log file. The declaration for this entry point looks like this:
void onLogRecord(
uint64_t timestamp,
uint64_t recordCode,
void const* p,
size_t size
) {
// TODO: handle new log records
}
It’s not so easy to memorize the signature for this function, so we made it possible to auto-generate it with a few keystrokes. Start typing the function name (letter o), then press Ctrl + Space, and an auto-completion list will pop up. Select onLogRecord (it will likely be the only item on the list):
Hit Enter, and the skeleton for your event handler will be auto-generated.
Let’s have a quick break down of parameters passed to onLogRecord.
Parameters:
| Parameter | Description |
|---|---|
| timestamp | Timestamp of the log record (see sys.getTimestamp). |
| recordCode | Record code. May be one of log.StdRecordCode or something transport-specific. |
| p | Pointer to the block of record-code-specific parameters. |
| size | Size of record-code-specific parameters in bytes. |
Often times it’s more convenient to use a typed pointer for p instead of void const* as to avoid additional type casts. You can use any non-thin data pointer type for p, for example:
void onLogRecord(
uint64_t timestamp,
uint64_t recordCode,
char const* p, // also OK
size_t size
) {
...
}
Basic Processing Of RX Stream
As the old saying goes, who says “A” must say “B”. Let’s write a script that will reply B for every A discovered inside the incoming (RX) data stream.
void onLogRecord( uint64_t timestamp, uint64_t recordCode, char const* p, size_t size ) { if (recordCode != log.StdRecordCode.Rx) return; // we only care about RX stream char const* end = p + size; while (p < end) { char const* a = memchr(p, 'A', end - p); if (!a) break; transmit("B"); p = a + 1; } }
To test this script, open a TCP Server on some local port (e.g. 8080), then start a TCP Connection and connect to 127.0.0.1:8080. Run this script on one side of this connection, and send some packets containing A-s from the other side. Note that you receive as many B-s back as there were A-s sent.
Buffering Commands
The formal definition of the “protocol” used in the example above is very simple – a command consists of a single character A, and it can occur anywhere in the incoming RX stream. As such, the code to process such commands is trivial.
All real-life protocols, are, of course, more complex than that. Commands typically occupy more than one byte. Also, the length of the packet is often times non-constant, but rather dependent on the contents of a particular packet.
Yet another complication to take care of is the fact that most transports – even reliable ones, such as TCP! – don’t guarantee the delivery of a packet as a whole. When you send 10 bytes on one end of a connection, you can’t expect to recv exactly 10 bytes on the other end. It can be delivered in chunks of, for example 6 and 4 bytes, or maybe 5, 3, and 2 bytes.
Bottom line – you cannot take a single log.StdRecordCode.Rx record and start analyzing its data while assuming it contains the whole packet. You have to buffer it first, and analyze it only after the whole packet has been buffered. How exactly to buffer a packet, depends on the protocol.
Plain-text Protocols
In text-based protocols, commands are usually terminated with carriage-return CR (\r) and/or line-feed LF (\n) characters. Let’s assume, CR (\r) is used as a terminator for a command. Then you can use the following boilerplate as a base for your script:
import "std_Buffer.jnc" void onLogRecord( uint64_t timestamp, uint64_t recordCode, char const* p, size_t size ) { if (recordCode != log.StdRecordCode.Rx) return; // we only care about RX stream static std.Buffer buffer; char const* end = p + size; while (p < end) { char const* cr = memchr(p, '\r', end - p); if (!cr) { // not yet buffer.append(p, end - p); break; } buffer.append(p, cr - p); // buffer the last chunk // process the buffered command // in this example, we simply write it to the log buffer.append(0); // ensure zero-termination g_logWriter.write($"command: '%s'"(buffer.m_p)); // clear buffer and move onto the next command buffer.clear(); p = cr + 1; } }
To make things more interesting, let’s assume that the command must be prefixed with STX (\x02). This is a pretty common pattern in Serial-based protocols.
import "std_Buffer.jnc" void onLogRecord( uint64_t timestamp, uint64_t recordCode, char const* p, size_t size ) { if (recordCode != log.StdRecordCode.Rx) return; // we only care about RX stream static std.Buffer buffer; static bool isCommand; char const* end = p + size; while (p < end) { if (!isCommand) { // skip everything until STX char const* stx = memchr(p, '\x02', end - p); if (!stx) break; // prepare buffer for the upcoming command p = stx + 1; isCommand = true; buffer.clear(); } char const* cr = memchr(p, '\r', end - p); if (!cr) { // not yet buffer.append(p, end - p); break; } buffer.append(p, cr - p); // buffer the last chunk // command is fully buffered, process it // in this example, we simply write it to the log buffer.append(0); // ensure zero-termination g_logWriter.write($"command: '%s'"(buffer.m_p)); // prepare for the next chunk of data p = cr + 1; isCommand = false; } }
Run this code and try sending it some data with “commands” embedded between STX and CR characters. Note that it works correctly no matter if you send those commands byte-by-byte, in chunks, or multiple commands in one block.
Using Jancy Regex Switches
Let’s make the previous example a little bit more realistic. After we’ve successfully accumulated the command until a CR character, let’s actually analyze it and do appropriate actions depending on its contents.
We’ll do so using the Jancy regex switch statement:
import "std_Buffer.jnc" void onLogRecord( uint64_t timestamp, uint64_t recordCode, char const* p, size_t size ) { if (recordCode != log.StdRecordCode.Rx) return; // we only care about RX stream static std.Buffer buffer; char const* end = p + size; while (p < end) { char const* cr = memchr(p, '\r', end - p); if (!cr) { // not yet buffer.append(p, end - p); break; } buffer.append(p, cr - p); // buffer the last chunk // process the buffered command using regex switch switch (string_t(buffer.m_p, buffer.m_size)) { // run on buffer case "about": transmit("Command server\r\n"); break; case "version": transmit("Version 1.0.0\r\n"); break; case "command([0-9]+)": transmit($"Command: $1\r\n"); break; default: transmit("Unknown command\r\n"); } // clear buffer and move onto the next command buffer.clear(); p = cr + 1; } }
The regex switch statement is compiled into a single DFA that will select the matching branch as efficient as possible. So it’s both the most natural and the most efficient way of analyzing text-based data.
Binary Protocols
In binary protocols, packets are normally prefixed with a fixed-length header which carry the information about the full length of the packet. Therefore, we first need to buffer RX bytes until we have the whole header, and then buffer RX bytes until we have collected the whole packet.
import "std_Buffer.jnc" struct PacketHdr { uint16_t m_code; uint16_t m_dataSize; // followed by command-specific data (m_dataSize bytes) } void onLogRecord( uint64_t timestamp, uint64_t recordCode, char const* p, size_t size ) { if (recordCode != log.StdRecordCode.Rx) return; // we only care about RX stream static std.Buffer buffer; char const* end = p + size; while (p < end) { // first, we need to buffer the header if (buffer.m_size < sizeof(PacketHdr)) { size_t leftoverRx = end - p; size_t leftoverHdr = sizeof(PacketHdr) - buffer.m_size; if (leftoverRx < leftoverHdr) { // not yet buffer.append(p, leftoverRx); break; } buffer.append(p, leftoverHdr); p += leftoverHdr; } // the header is ready, buffer the rest of the packet PacketHdr const* hdr = (PacketHdr const*)buffer.m_p; size_t leftoverRx = end - p; size_t leftoverPacket = hdr.m_dataSize - (buffer.m_size - sizeof(PacketHdr)); if (leftoverRx < leftoverPacket) { // not yet buffer.append(p, leftoverRx); break; } buffer.append(p, leftoverPacket); // buffer the last chunk // packet is fully buffered, process it // in this example, we simply write command info to the log g_logWriter.write($"command code: %d, data: %d bytes"( hdr.m_code, hdr.m_dataSize )); // prepare for the next chunk of data buffer.clear(); p += leftoverPacket; } }