Posts made by Vladimir

With IO Ninja, you can monitor Modbus TCP, analyze communications, and export data for further processing.

However, there's no out-of-the-box integration with data acquisition services.

If it's the monitor-and-analyze part you are interested in, then the roadmap is as follows.

The first stage is to acquire the TCP conversation between Modbus Master and Modbus Slave. In IO Ninja, this can be done via two alternative approaches.

(a) You can capture Ethernet packets using Ethernet Tap or Pcap Sniffer (capturing via Pcap could require some trickery if the Modbus Master is not running on the same workstation as IO Ninja), then reconstruct TCP conversation by attaching TCP Flow Analyzer.

(b) But a more straightforward approach would be to use IO Ninja as a man-in-the-middle and let it forward TCP traffic between the Master and Slave using TCP Proxy. Whenever this man-in-the-middle approach is applicable (i.e., you are in control of the Master and can redirect it to IO Ninja), that should be your choice -- it's much more straightforward and reliable.

After you set everything up for reconstruction of the raw TCP conversation between Master and Slave, you attach the Modbus Analyzer to decode the raw bytes into human-readable Modbus frames.

IO Ninja log is very powerful and should be more than enough to inspect and analyze the Modbus conversation.

If you need to post-process it further, one option would be to create a Log Layer to do the job (remember, IO Ninja is scriptable!). However, for people who have never touched IO Ninja scripting before, a much more straightforward path would be to save the log as .njlog and use any scripting language of their choice to walk over it.

The .njlog file format is very simple and all the relevant data structures & constants are open-source and defined in scripts/api/log_RecordCode.jnc and scripts/api/log_RecordFile.jnc (included in IO Ninja packages).

Hope this helps; feel free to follow up with any questions!

Hello Ivan,

capture serial communication with up to 3.3mbps

3.3 Mbps is beyond the maximum baud rate supported by Serial Tap. Baud rates <= 1 Mbps are guaranteed to be reliably captured; ~2 Mbps could work in theory, but errors are possible, and I think 3 Mbps would be outright rejected by the UART controller we use in Serial Tap.

add custom protocol parsing logic

This is possible; IO Ninja supports so called "layer" plugins that can transform original logs and add decoded protocol messages. See the Modbus Analyzer for an example of such layer plugin (it's open-source and located at /scripts/plugins/Modbus/)

perform time event measurement

Events are timestamped on the PC side -- the Serial Tap reports raw bytes and status line changes, IO Ninja adds timestamps and writes them to log.

be controlled by Python with ability to get parsed data, set time markers ad get time for each events

If you need to implement custom processing in Python, you can use ioninja-hwc (https://ioninja.com/features/hardware-client.html#cli) -- capture Serial Tap events from the command line and redirect those to a file or stdout; then implement whichever custom logic you need in Python.

Unfortunately, I can't see the wiring or the mode switch position from the photo. However, on the screenshot, all the data is RX (green). Most likely, it's the usual two-wire RS485 (half-duplex), but the Modbus Analyzer is configured for full-duplex (TX Master, RX Slave) -- which is causing those parsing errors.

We have a KB article on configuring Modbus for half-duplex links:
https://ioninja.com/doc/kb/modbus_half_duplex.html

Please let me know if this works for you.

@gary-biagioni

I guess the above just goes into the scripting panes. One problem I have is how do I debug the program I write. Not sure how to be able to do that, for example, look at the contents of a variable.

You can use the good old printf debugging.

import "hexEncoding.jnc"

void main() {
	int a = 10;
	char s[] = "abcdef";
	char buf[] = 0x"01 02 03 04 05";
	
	string_t msg = $"a: %1/0x%(1;02x) s: %2 buf: %3"(a, s, encodeHexString(buf, sizeof(buf), ' '));
	// string_t msg = $"a: %d/0x%02x s: %s buf: %s"(a, a, s, encodeHexString(buf, sizeof(buf), ' ')); // same
	printf($"This will go to the system log: $msg\n");
	g_logWriter.write(log.StdRecordCode.PlainText, $"This will go to the normal log: $msg\n");
}

The system log can be viewed via Menu->View->System Log

@gary-biagioni

That said above this looks super interesting to handle responses

https://ioninja.com/doc/developer-manual/tutorial-ias-server.html

At the moment of writing this tutoiral, onLogRecord was the only approach to reading incoming data bytes.

Of course, it still works, but we now have receive and receiveAll functions. I believe they are much easier to use.

Hi Gary,

Sorry, the API documentation is currently just a placeholder; it's auto-generated from the script sources (you can also check those sources directly at $IONINA_DIR/scripts/api/). However, we also have some scripting tutorials in the Developer Manual (https://ioninja.com/doc/developer-manual/tutorials.html) -- those should be helpful.

For the in-app-scripting inside the "Script" pane, all the available function declarations can be found at scripts/api/ias.jnc

To answer your question, the receive and receiveAll functions are used to collect raw data bytes that appear in your log as the RX (incoming) stream.

receive is the most basic method; it accepts the buffer and timeout and works as such:

1. if there are any incoming data bytes at the moment of call -- those data bytes are returned immediately;

2. otherwise, receive waits for incoming data for at most timeout milliseconds before returning. If a chunk of incoming data arrives before timeout expires, it will be returned as soon as received. If timeout expires and no data arrives -- 0 will be returned. timeout defaults to -1 which is equivalent to infinite wait for data (receive will not return until at least one byte of incoming data arrives).

receiveAll is a helper wrapper around receive and is used to fill the supplied buffer entirely (you can see its implementation in ias.jnc). This is convenient when your script needs to fill a fixed-size packet header before proceeding. If a packet arrives in chunks, the receiveAll will wait until the buffer is complete and only then return. The overloaded version of receiveAll with the timeout parameter will attempt to fill the buffer entirely -- but will bail after timeout milliseconds.

Hope this helps; feel free to ask more.

Are those little icons on the snapshot indicates the type of connection besides it is a pipe? (Socket, shared memory, message queue)

Pipe Monitor shows named and anonymous pipes only.

Anonymous pipe opens will be marked as (unnamed). If you run a Win32 code to create anonymous pipes such as:

	HANDLE hReadPipe;
	HANDLE hWritePipe;
	dword_t actualSize;
	char data[] = "abcdefghi";
	char buffer[1024];

	::CreatePipe(&hReadPipe, &hWritePipe, NULL, 0);

	::WriteFile(hWritePipe, data, sizeof(data), &actualSize, NULL);
	::ReadFile(hReadPipe, buffer, sizeof(buffer), &actualSize, NULL);

	::CloseHandle(hReadPipe);
	::CloseHandle(hWritePipe);

You should see something like:

The screenshot failed to upload; could you try again, please?

Regarding the IPC method, well, the Pipe Monitor shows communications over named and anonymous pipes. To see if a particular read or write is issued over an anonymous or named pipe (and the name of the pipe), you have to follow the log up all the way to the pipe open operation for this particular file ID; there you'll see the file name and the role (client or server).

Yes, you can use Serial Tap to monitor RS422 communications. Use the RS485 portion of the terminal block to connect your TX+/- and RX+/- lines.

Hello,

ModbusRtuWritePacket is from the legacy Modbus packet template library; it doesn't support multiple values per packet. The new Modbus plugin uses structures defined in scripts/protocols/io_Modbus.jnc. If you need to do it programmatically, just assemble your packet structure from necessary chunks (ADU hdr, PDU hdr, function-specific params, values, CRC). Important -- be sure to add pragma(Alignment, 1) as to avoid unintended struct paddings!

After the packet structure is defined, fill in the fields and calculate the checksum. Modbus RTU checksum is CRC16 ANSI (with init 0xffff) of the whole frame excluding the checksum itself.

The full script listing is below:

import "io_Modbus.jnc"

void main() {
	enum {
		DeviceAddress   = 1,
		RegisterAddress = 0xA000,
		RegisterCount   = 2,
	}

	pragma(Alignment, 1)
	struct MyPacket:
		io.ModbusRtuAduHdr,
		io.ModbusPduHdr,
		io.ModbusWriteMultipleParams {

		bigendian uint16_t m_registers[RegisterCount];
		uint16_t m_crc;
	}

	MyPacket packet;
	packet.m_deviceAddress = DeviceAddress;
	packet.m_func = io.ModbusFunc.WriteMultipleRegisters;
	packet.m_address = RegisterAddress;
	packet.m_count = RegisterCount;
	packet.m_size = sizeof(packet.m_registers);
	packet.m_registers[0] = 123;
	packet.m_registers[1] = 456;
	// ...
	packet.m_crc = crc16_ansi(packet, offsetof(packet.m_crc), -1);

	transmit(packet, sizeof(packet));
}

Hello Jose,

Sure, no rush with the feedback -- please take your time.

And thank you so much! Wishing you a Merry Christmas and a Happy New Year as well! Enjoy your winter holidays!

Here's an internal build of IO Ninja for arm64:

https://tibbo.com/downloads/archive/ioninja/.internal/prerelease/ioninja-5.7.1-c-linux-arm64.tar.xz

The ioninja-hwc introduces two new parameters:

--split-size <bytes> and --split-time <seconds>

Both options can be used together; in this case, the splitting will occur based on whichever condition is met first. Both options support reasonable suffixes, e.g., --split-size=10M, --split-time=6h, etc.

Let me know if this works for you!

If this is something you could add "for free" to your new development, great! If not, well, I think that this is no a common case, in fact, I found it very weird (but "real" indeed).

Yes, it comes for free, so we'll have a setting for that!

if possible/feasible/reasonable, consider the options of splitting by size (eg. each "n" MB) and by time (eg. each "t" minutes). It is just an idea... perhaps it seems more logic just consider the size, but -eg. for my case- it is more convenient to have captures of info more or less time aligned (eg. daily aligned... in fact, I am currently splitting them each 6 hours)

Good point! Let's have settings for both!

consider the option of being "transparent" when power-off...

Oh yes, I agree! This is absolutely one of the must-have features for the next generation of Ethernet Tap. Disconnecting power from the tap should not disrupt the existing link.

Well, then have to add help tooltips to every other setting, too, for consistency... Which could be the right thing to do, actually

In this particular case, however, I'm kinda inclined to simplify and drop this setting altogether (use hex-line-size / 2 for this special dual-hex-view mode).

Hello,

The Serial Tap for IO Ninja (and the majority of standard USB-to-Serial adapters) does not support 9-bit UART communications. The upcoming Serial Tap Pro (to be released early next year) will support this feature natively, but in the meantime, here’s a workaround.

First, some background. The most common use case for the 9th bit in UART is marking the address byte in some industrial automation protocols. This important optimization enables slave nodes to inspect only the first (address) byte; if the address does not match, the slave can safely ignore all subsequent bytes until the next address byte (i.e., the next UART frame with the 9th bit set).

In these scenarios, the 9th bit is clear for most payload bytes and set only for address bytes. To monitor such communications using a Serial Tap (or a USB-to-RS485 adapter), use the settings as follows:

Baud rate: <specify-the-correct-baud-rate>
Data bits: 8
Parity: Space
Stop bits: 1

With these settings, address bytes will generate PARITY errors (the parity is set to "space," but the parity bit is 1), while the rest of the traffic will remain free of line errors.

Reporting of line errors is inherently asynchronous, so the PARITY errors won't necessarily align precisely with each address byte (BTW, Serial Tap Pro will address this, too, and line errors will be byte-precise). Still, you should see a PARITY error somewhere around the address byte.

Hope this helps; let me know if this workaround works for you!

Hello,

It's not very clear what you mean by "automatically labeling log files according to the session name or device ID". When you create a new IO Ninja session, the log is stored in a temporary file. When you save the session, you explicitly specify a directory name to store the session data (or a single file name if you only need to save the log file) -- so you are the one picking the file names.

If you need to post-process the log files externally, you can do it in any language of your choice (Python, JavaScript, C, etc.) The .njlog file format is very simple. A brief overview can be found here: https://ioninja.com/doc/developer-manual/logging-engine.html) and all the internal structures are open-source; see scripts/api/log_RecordCode.jnc and scripts/api/log_RecordFile.jnc for all the relevant declarations.

Let me know if you have any follow-up questions regarding the file format.

Apologies for the inconvenience.

Please restart the Pipe Monitor session and set the Filter to "None" if you don't need to filter by name. Alternatively, you can set the name wildcard to * (i.e., an asterisk).

Tech details:

The latest release of IO Ninja contains a regression that results in an exception when setting an empty wildcard as a filter; the log engine remains suspended after this exception, thus resulting in the "empty log" you saw.

Here's how to fix it (until the next release takes care of it):

1. Open <ioninja-dir>/scripts/common/log_MonitorFilter.jnc in any text editor
2. Search for MonitorFilter.setFilter
3. Add the following snippet right before processing the wildcard string:

switch (filterKind) {
case MonitorFilterKind.FileName:
case MonitorFilterKind.ProcessName:
	// <<<<<<<
	if (!filter) {
		m_filterKind = MonitorFilterKind.None;
		break;
	}
	// >>>>>>>

Save the script and restart the Pipe Monitor session (no need to restart the app).

Please let me know if it works for you.

This was done for the I2C/SPI Tap. The IO Ninja log engine supports this special dual-hex view for SPI which looks like this:

But I totally see how this could be confusing -- it's only relevant to this single plugin and a single mode in this plugin!

Maybe, we should remove this setting completely and simply calculate it as hex-line-size / 2?