Dear friends!

It has been a long time since we released an update for IO Ninja -- more than eight months! Such a delay happened because we were working hard on the new major release, IO Ninja 5. And I'm happy to announce that it's finally out!

The most significant change from the previous generation of IO Ninja is the introduction of accounts, capabilities, and subscriptions. Please read the intro on our website to learn more about our motivation for such a major change. The dedicated section of the user manual covers this new ecosystem in more detail.

The IO Ninja 5 also features two new plugins dedicated to helping web developers debug modern WebSocket-based applications. Please read more about these plugins on the respective pages: WebSocket Client and WebSocket Server.

IO Ninja 5 includes a critical update of the Device Monitor Service for Linux that enables running the Serial Monitor on modern Linux kernels. There's also a major clean-up of UI for the macOS version, critical bug fixes and optimizations in the logging engine, and plenty of other important updates and improvements.

Besides the software updates, with IO Ninja 5, we introduce a new e-store that seamlessly integrates with the main website and provides a much more smooth experience. And of course, we -- finally! -- opened this forum, just as so many of you suggested. Here, you can ask questions, share your suggestions on improving IO Ninja, and receive support -- both from the IO Ninja Team and other users.

We worked with passion and dedication to release this new version of IO Ninja. We are sure that IO Ninja will help you in your work, and you will enjoy using it!

Currently, IAS (in-app-script) can only control the underlying session via:

• connect()
• disconnect()
• transmit(p, size)

Fine-tuning of the sessions configuration is currently not possible from IAS. But we plan to add this capability in the near future.

Each session class will export a public IAS interface (e.g., a serial session will have properties to query and control baud rate, parity, status lines, etc.) Then, IAS should be able to access this interface via a global constant g_session (or something like that).

Hello Bartosz,

The script for a filter to do what you want is very simple. First, you deduce the state of the CS line (from the I2cSpiTapLogRecordCode.SpiStart and SpiStop log records); then, you use this state to either hide or show the MOSI/MISO data (the log.StdRecordCode.TxRx log records).

The source code for such a filter might look something like this:

import "doc_Layer.jnc"
import "I2cSpiTap/I2cSpiTapLogRecordCode.jnc"

class SpiCsFilterLayer:
	doc.Layer,
	log.Filter
{
protected:
	ui.EnumProperty* m_csFilterProp; // a property to choose the filtering strategy
	int m_cs; // the state of the CS line

public:
	construct(doc.PluginHost* pluginHost);

	override bool filter(
		uint64_t timestamp,
		uint64_t recordCode,
		void const* p,
		size_t size
		);
}

SpiCsFilterLayer.construct(doc.PluginHost* pluginHost)
{
	basetype.construct(pluginHost);

	ui.EnumPropertyOption csFilterOptions[] = {
		{ "Show always", -1 },
		{ "Show when CS low", 0 },
		{ "Show when CS high", 1 },
	}

	m_csFilterProp = m_pluginHost.m_propertyGrid.createEnumProperty(
		"MOSI/MISO filter",
		"Show MOSI/MISO filtering criteria",
		csFilterOptions,
		countof(csFilterOptions)
	);

	m_cs = -1; // -1 means unknown
	pluginHost.m_log.addFilter(this);
}

bool SpiCsFilterLayer.filter(
	uint64_t timestamp,
	uint64_t recordCode,
	void const* p,
	size_t size
	)
{
	bool isVisible = true;

	switch (recordCode)
	{
	case log.StdRecordCode.SessionStarted:
		m_cs = -1; // reset to unknown
		break;

	case I2cSpiTapLogRecordCode.SpiStart:
		m_cs = 0;
		break;

	case I2cSpiTapLogRecordCode.SpiStop:
		m_cs = 1;
		break;

	case log.StdRecordCode.TxRx:
		isVisible =
			m_csFilterProp.m_value == -1 || // show always
			m_csFilterProp.m_value == m_cs; // matches the current state of CS
		break;
	}

	return isVisible;
}

An archive with the complete filter plugin is attached (SpiCsFilter.7z)

Usage:

1. Start I2C/SPI session
2. Open the SpiCsFilter.njplg file to attach this filter
3. Go to "Settings"
4. Select the appropriate filtering criteria
5. Hit "Apply and rebuild log" to apply the filter to the whole log (clicking "Apply" or "OK" will apply the filter "from now on" (i.e., to the follow-up log records only)

A little bit late for a follow-up, but still Which OS are you running on? Just FYI, ioninja-5.2.0 now picks up the system-wide dark theme on macOS and Linux with KDE desktops. If you running one of those, you can try the newly added dark theme support io IO Ninja. If not, stay tuned, as we do plan to put some extra effort and allow switching IO Ninja to dark mode at the user's will.

Hello,

Apologies for the delayed response. Support for njlog output directly from ioninja-hwc is on our TODO list (it was already requested by users). For the time being, you need to post-process the raw output of ioninja-hwc using any scripting language of your choice.

The protocol of communication between ioninja-hwc and plugins such as Serial over SSH or Serial Tap over SSH is not currently documented, but the decoding process and all relevant constants and data structures can be looked up in the sources of those plugins (remember, all IO Ninja plugins are open-sourced and available at ioninja/scripts/plugins).

In particular, all the relevant constants and packet structures for the ioninja-hwc protocol are contained in ioninja/scripts/common/io_HwcProto.jnc. The output file generated by ioninja-hwc is basically a sequence of "out" messages of this protocol. Each message starts with HwcMsgHdr followed by extra data block, the meaning of which depends on the message code. For example, HwcMsgCode.Rx is followed by the received bytes, HwcMsgCode.SerialTapCtsDsr is followed by HwcSerialStatusLines flags, etc.

Try decoding the output of ioninja-hwc using any language of your choice, and feel free to let me know if you run into a stumbling block or have any other questions.

Thanks for the feedback!

Auto-baud rate could be convenient, but when used in a sniffer, there's a quirk -- detecting baud rate can't be 100% reliable without prior knowledge of the incoming data stream. Serial protocols that support auto-baud rate always use specific packet headers that allow reliable detection of the baud rate for this exact reason. So, a sniffer with auto-baud rate detection (which can't make any assumptions about the underlying data) could initially produce incorrect bytes before it deduces the actual baud rate.

All that said, yes, we do consider adding the automatic baud rate detection in the new generation of our Serial Tap devices.

Not exactly a bug but rather a missing misuse protection.

So, what's happening? The automatic calculation of Modbus TCP frames is done like this:

void updateModbusTcpLength(void* p) {
	size_t size = dynamic sizeof(p);
	if (size < sizeof(io.ModbusTcpAduHdr))
		return;

	((io.ModbusTcpAduHdr*)p).m_length = size - offsetof(io.ModbusTcpAduHdr.m_deviceAddress);
}

Basically, m_length is set to the length of the payload (from m_deviceAddress and to the very end of the packet).

In your case, the packet length is 17, offset of m_deviceAddress is 6, so m_length is set to 11. However, the size of Modbus TCP read should always be 12 (not 17). If you erase the trailing 5 bytes from your packet, it will work as expected.

That should solve the issue; for more relevant details, read on.

By design, packet templates never allow creating a packet shorter than the template itself. However, it's allowed to add any suffix after the templated headers (that would be required for Modbus TCP write, for instance).

In ioninja-5.1.0 we added the fixedSize attribute exactly to prevent unintentional misuse in case of packets that should never grow longer than the template itself.

Modbus TCP read is exactly this kind of packet, but in the current version of IO Ninja, it's missing the fixedSize attribute. We will add it in the next release of IO Ninja, but in the meantime, you can fix it by yourself. To do so, please open ioninja/scripts/packets/ModbusTcp.jnc in any text editor, locate the definition of ModbuTcpReadPacket (in the very beginning), and modify it as such:

[ 
	displayName = "Modbus TCP read",
	fixedSize
]
struct ModbusTcpReadPacket {
// ...

In order to apply it, please reload this template into the Packet Template editor (Edit packet template -> Load stock script -> Modbus TCP).

Let me know if this works for you.

Confirmed. There's a regression in the Ethernet Tap plugin on Linux builds that somehow got under the radar during routine pre-release testing; will be fixed in the very next release.

Workarounds:

1. use ioninja-hwc to capture to .pcap from command line:

$ ./ioninja-hwc --ethernet-tap --pcap --out=my-capture.pcap

2. use the previous release of ioninja: https://tibbo.com/downloads/archive/ioninja/ioninja-5.5.1/

3. if that's possible in your case, use the windows or macos builds of the latest ioninja-5.6.0 (the regression only affects Linux builds).

Also, is it mandatory to be root, or is there a way to allow a user to open the Ethernet tap ?

You can add a UDEV rule to assign less restrictive permissions to USB devices based on VID/PID:

https://ioninja.com/doc/kb/linux_usb_permissions.html

For the Ethernet Tap, use these parameters:

SUBSYSTEM=="usb", ATTR{idVendor}=="326f", ATTR{idProduct}=="0003", MODE="0666", SYMLINK="ethernet-tap"

Here's an internal build of IO Ninja for arm64:

https://tibbo.com/downloads/archive/ioninja/.internal/prerelease/ioninja-5.7.1-c-linux-arm64.tar.xz

The ioninja-hwc introduces two new parameters:

--split-size <bytes> and --split-time <seconds>

Both options can be used together; in this case, the splitting will occur based on whichever condition is met first. Both options support reasonable suffixes, e.g., --split-size=10M, --split-time=6h, etc.

Let me know if this works for you!

If this is something you could add "for free" to your new development, great! If not, well, I think that this is no a common case, in fact, I found it very weird (but "real" indeed).

Yes, it comes for free, so we'll have a setting for that!

if possible/feasible/reasonable, consider the options of splitting by size (eg. each "n" MB) and by time (eg. each "t" minutes). It is just an idea... perhaps it seems more logic just consider the size, but -eg. for my case- it is more convenient to have captures of info more or less time aligned (eg. daily aligned... in fact, I am currently splitting them each 6 hours)

Good point! Let's have settings for both!

consider the option of being "transparent" when power-off...

Oh yes, I agree! This is absolutely one of the must-have features for the next generation of Ethernet Tap. Disconnecting power from the tap should not disrupt the existing link.

Well, then have to add help tooltips to every other setting, too, for consistency... Which could be the right thing to do, actually

In this particular case, however, I'm kinda inclined to simplify and drop this setting altogether (use hex-line-size / 2 for this special dual-hex-view mode).

Hello,

The Serial Tap for IO Ninja (and the majority of standard USB-to-Serial adapters) does not support 9-bit UART communications. The upcoming Serial Tap Pro (to be released early next year) will support this feature natively, but in the meantime, here’s a workaround.

First, some background. The most common use case for the 9th bit in UART is marking the address byte in some industrial automation protocols. This important optimization enables slave nodes to inspect only the first (address) byte; if the address does not match, the slave can safely ignore all subsequent bytes until the next address byte (i.e., the next UART frame with the 9th bit set).

In these scenarios, the 9th bit is clear for most payload bytes and set only for address bytes. To monitor such communications using a Serial Tap (or a USB-to-RS485 adapter), use the settings as follows:

Baud rate: <specify-the-correct-baud-rate>
Data bits: 8
Parity: Space
Stop bits: 1

With these settings, address bytes will generate PARITY errors (the parity is set to "space," but the parity bit is 1), while the rest of the traffic will remain free of line errors.

Reporting of line errors is inherently asynchronous, so the PARITY errors won't necessarily align precisely with each address byte (BTW, Serial Tap Pro will address this, too, and line errors will be byte-precise). Still, you should see a PARITY error somewhere around the address byte.

Hope this helps; let me know if this workaround works for you!

Hello,

It's not very clear what you mean by "automatically labeling log files according to the session name or device ID". When you create a new IO Ninja session, the log is stored in a temporary file. When you save the session, you explicitly specify a directory name to store the session data (or a single file name if you only need to save the log file) -- so you are the one picking the file names.

If you need to post-process the log files externally, you can do it in any language of your choice (Python, JavaScript, C, etc.) The .njlog file format is very simple. A brief overview can be found here: https://ioninja.com/doc/developer-manual/logging-engine.html) and all the internal structures are open-source; see scripts/api/log_RecordCode.jnc and scripts/api/log_RecordFile.jnc for all the relevant declarations.

Let me know if you have any follow-up questions regarding the file format.

Apologies for the inconvenience.

Please restart the Pipe Monitor session and set the Filter to "None" if you don't need to filter by name. Alternatively, you can set the name wildcard to * (i.e., an asterisk).

Tech details:

The latest release of IO Ninja contains a regression that results in an exception when setting an empty wildcard as a filter; the log engine remains suspended after this exception, thus resulting in the "empty log" you saw.

Here's how to fix it (until the next release takes care of it):

1. Open <ioninja-dir>/scripts/common/log_MonitorFilter.jnc in any text editor
2. Search for MonitorFilter.setFilter
3. Add the following snippet right before processing the wildcard string:

switch (filterKind) {
case MonitorFilterKind.FileName:
case MonitorFilterKind.ProcessName:
	// <<<<<<<
	if (!filter) {
		m_filterKind = MonitorFilterKind.None;
		break;
	}
	// >>>>>>>

Save the script and restart the Pipe Monitor session (no need to restart the app).

Please let me know if it works for you.

This was done for the I2C/SPI Tap. The IO Ninja log engine supports this special dual-hex view for SPI which looks like this:

But I totally see how this could be confusing -- it's only relevant to this single plugin and a single mode in this plugin!

Maybe, we should remove this setting completely and simply calculate it as hex-line-size / 2?

Hello again, Josep,

Yes, it's in the works, but it's currently sidelined in favor of the upcoming release of Serial Tap Pro -- which I mentioned in my response to your post about the TTL signal inversion.

Full-scale development of the new generation of Ethernet Tap will be resumed after that.