administrators

I think it would be a very helpful feature to add to IO ninja to just be able to save a log as a raw binary file for each direction.

It's very easy to achieve with a log filter/observer plugin script.

However, I can't say I find it very useful to have two raw binary files containing just the raw bytes flowing in each directing -- how would you reconstruct the mapping between requests and replies in case of some non-trivial conversation between the client and server?

I often have serial port logs that I just want to save to a bin file as-is, and I end up closing IO ninja and just going back to something like CoolTerm which supports this natively.

If it's a uni-directional log kind of thing (e.g., your device keeps dumping debug printfs to a RS232, and you want to save it to a file) -- this can be done using Session Linking. Open a Serial session, open a File Stream session in Write-only mode and point it to some file, then "link" those sessions. Everything that the Serial session reads from the device, will be redirected to your file.

it also supports recording directly to a file, in case the computer crashes or powers off during logging

Actually, IO Ninja always keeps its logs in disk files. When you "Save Log", the temporary .njlog file simply gets copied into the file you select. But if you "Save Session", those .njlog files will get stored in the specified folder, and you will know exactly where to find them in case of a crash.

But once again, a dedicated plugin that would explicitly save RX to one file and TX to another is quite easy to implement. I don't know, maybe we should add it to one of the upcoming releases as an official plugin (or a sample)?

Try the updated analyzer:

BacNetMsTp.7z

It checks CRCs now and discards frames with broken headers (in the previous log, the decoder didn't actually stop, but because a broken header specified a very long payload size, it kept buffering data assuming it still was a payload).

Also, when you upload the log, please detach the Analyzer before saving the log. This way, we can access the original raw & unprocessed data (instead of the already decoded frames).

No prob, I've increased the upload file size limit to 8MB.

Also, you could have archived it with 7z

Apparently, this was a permission issue; it doesn't apply to administrators, so I didn't even know it was there...

Anyway, I've adjusted the file upload permissions for registered users; please try again.

Tried using the upload images(right most icon) to send a 7z zip file but I don't have privileges for it.

Not "Upload image", "Upload file". Does your "Reply toolbar" have this button?

How do I send you the log that is out of sync

Hit "Reply" (not "Post quick reply"); the rightmost button on the toolbar is "Upload File". 7-zip it first so that it has the .7z extension permitted by this forum.

Removing the padding gives this trying to load the Analyzer:

Most likely, you just commented out the padding code, but forgot to add a semicolon ; at the end of the statement. In Jancy, just like in C, declarations and expression statements must be terminated with a semicolon.

While using this analyzer should we have been running Settings->Log Engine->Binary Data->Binary Data Merge with a 3msec threshold.

This won't affect the analyzer, but I think it's completely unnecessary now. You will see packets boundaries anyway -- and not just boundaries, decoded header fields, too.

P.S.
For simplicity, just sync on 0x55; that should be good enough.

In BacNetMsTpParser.parse(...), add this:

	while (p < end) {
		void const* p0 = p;

		// 0. sync on 0x55

		if (!m_buffer.m_size) {
			void const* preamble = memchr(p, 0x55, end - p);
			if (!preamble)
				break;

			p = preamble; // skip everything before the preamble
		}

		...

Version 5.5.0 will not display the contents of BACnet headers properly (after you click [+] to expand). The pre-release .7z file is a portable archive; extract it anywhere and run ioninja-5.5.1\bin\ioninja.exe.

Re "stop" of the analyzer -- could you share the original .njlog with the large packet?

I suspect that the reason is the incorrect handling of padding in the script (it went out of sync immediately after the large frame). Try removing the padding:

size_t frameSize =
	sizeof(BacNetMsTpHdr) +  // header
	hdr.m_length +           // payload
	sizeof(uint16_t);        // CRC
	// (hdr.m_length & 1);      // padding <---

Also, it makes sense to wait and sync on \x55\xff before starting buffering the packet; this way, the parser will re-sync after synchronization is lost (this still can occasionally happen because RS485 does not guarantee lossless delivery).

P.S. FYI, just as with any representation generated by log.representStruct(), you can click on a header field, and the corresponding data bytes will be highlighted below -- so that you can see the mapping between header fields and data bytes.

How do you get to the Transmit pane if it is greyed out(un-selectable) ?

The Transmit pane is greyed out in the Serial Tap plugin because it's a read-only sniffer device. You can transmit using the Serial Terminal plugin and a regular USB-to-RS485 adapter.

I tried using RegEx Markup with default Session settings and it does not want to break the packets up.

After adding "Force Latin-1 encoding" to the Log Regex Markup settings I'm getting color highlights but no new timestamps when it see's the pattern.

Right, I should have mentioned that Latin-1 encoding might be necessary when the pattern contains invalid UTF-8 sequences (such as \x55\xff).

In general, using timestamps to split the data stream into packets is not a reliable enough approach (unless every captured byte is precisely timestamped). Serial Tap doesn't do that; instead, timestamps are assigned to the whole blocks of data as they are received over USB. There are multiple layers of buffering involved (at both the Serial Tap and the PC side), so the timing could be distorted to some extent, and bytes from different packets could end up being read within the same block. It's much better to parse the stream and split it into packets based on the actual data contents.

I've created a draft of the BACnet MSTP protocol analyzer for you: https://tibbo.com/downloads/archive/ioninja/.internal/scripts/BacNetMsTp.7z

It breaks the stream into BACnet frames, gives a human-readable representation of the frame header, and highlights the payload (if any).

Feel free to modify or extend it to your liking. However, please note, that ioninja-5.5.0 has a regression that prevents log.representStruct() used in this script from operating properly (it's already fixed but the service release is not out yet). So, to use this plugin, please either (a) roll back to ioninja-5.4.2 or (b) use the internal build ioninja-5.5.0-a: https://tibbo.com/downloads/archive/ioninja/.internal/prerelease/ioninja-5.5.0-a-windows-amd64.7z

Let me know if it works for you, and feel free to ask me anything about the internal implementation of the analyzer script.