From the first screenshot, I assume, you are running on Windows, correct? Then I will use Windows-style paths and declarations in the samples below.

1. Command line

Yes. You can open the necessary plugin(s) and start capturing from the command line. There are multiple ways of achieving that, and the easiest would be this.

First, configure the session the way you need -- i.e., start the main plugin (e.g., Serial Tap), attach required analyzer plugins (e.g., Modbus RTU, or your own analyzer), and configure all the necessary settings using the UI. Then, save the session somewhere (e.g., C:/Projects/playground/ioninja/my-analyzer-session/my-analyzer-session.njssn)

Now, you can start IO Ninja passing it a path to .njssn to automatically open this session. To automatically start capturing, also pass -c (i.e., connect):

ioninja -c <path-to-njssn>

2. FFI (Foreign Function Interface)

Yes. One of Jancy design goals was easy and efficient interoperability with C/C++ code. Again, multiple ways of calling external code exist, the most straightforward would be declaring an interface to your C/C++ library via the dynamiclib construct.

Let's say, you have a library called my_lib.dll with the following exported C function:

extern "C"
__declspec(dllexport) 
size_t 
my_inverse(
	void* out,
	size_t outSize,
	const void* in,
	size_t inSize
) {
	// inverse the in-buffer and write it to out-buffer
}

In your Jancy code, declare an interface to this library like this:

pragma(ThinPointers, true);

dynamiclib MyLib {
	size_t 
	my_inverse(
		void* out,
		size_t outSize,
		const void* in,
		size_t inSize
	);
}

pragma(ThinPointers, default);

Basically, function declarations are the same as in C/C++; you only have to make sure to use "thin" pointers.

Jancy pointers are "fat" -- they include extra information about region, type, etc. To achieve ABI compatibility with C/C++, we need to explicitly declare pointers as "thin", like this:

char thin* p;

If there could be many FFI declarations -- like in an interface to a C/C++ library -- it may be more convenient to implicitly treat ALL pointers as "thin" using pragma(ThinPointers, true).

Now, when you need to call your library, load and call it like this:

MyLib myLib;
myLib.open("path-to-your-lib/my_lib.dll");
myLib.lib.my_inverse(p1, size1, p2, size2);

3. Creating an external process is possible, too, of course. However, passing binary data to a process via the command line -- and especially getting an answer back -- would be much less convenient and efficient. If you need to perform some data processing in external code, I strongly recommend using the method shown above, i.e., create a C/C++ library and call it from Jancy. It's both easy to implement and as efficient as it gets in runtime.

Hope this helps!

1. We would like to add this tool as part of our nightly build and validation process. Once a new build is created and deployed to a target, we'd like a way to run the IONinja via a batch file (running some command lines), load the protocol analyzer open the port and start recording the session. Is it possible to run the IONinja tool via an external API /commandline ? if so, can you send me an example how ?
2. The protocol analyzer is written in Jancy language. Is there an ability to access in Jancy APIs from DLLs created in C++ or Python ?
3. Can you activate in Jancy a batch file and do some manipulation on the bytes received ? i.e. pass the received bytes as an argument to a batch file, and get an answer from the batch file ?
   Many thanks
   Chaim

Just FYI, here's the added part (special handling for struct fields):

/// ...
		if (field.m_type.m_typeKind == jnc.TypeKind.Struct) {
			representation.addHyperText($"%1%2:\t"(indent.m_sz, displayName));
			representStruct(
				representation,
				dynamic (jnc.StructType*)field.m_type,
				displayName,,
				p + field.m_offset,
				baseOffset + field.m_offset,
				0, 0, // no [+/-] header, always expanded
				indentLevel + 1
			);

			continue;
		}
/// ...

Having sub-fields collapsible is theoretically possible but won't scale well -- we have 32 total fold-flags per record, so it's better to assign those statically (rather than from a loop in a recursive function). But I think having struct fields always expanded should be good enough in most cases.

Glad to hear you've made some good progress with your protocol analyzer!

Yes, log.representStruct should simplify and at the same time "standardize" representation of binary-based packets. And yes, you are right -- at the moment, it doesn't recursively represent all the nested structs. That's because this functionality was never required in the stock protocol analyzers in IO Ninja -- we use "layered" representation rather than "nested" because (a) this makes resulting tables more compact and easier to navigate, and (b) allows us to colorize layers using different colors. But there's nothing that prevents adding the support for nesting to log.representStruct; I think we can add that to the upcoming release.

How deep is the nesting in your case? Could you please provide definitions of the packet structs in your protocol?

Just FYI, the implementation of log.representStruct is in scripts/common/log_RepresentStruct.jnc; you can try experimenting with it by yourself until the official release.

Ok, to me it was very unclear, and took me a while to figure that out...

Installing and configuring NetBeans is an extra step, true. But I thought the explanation on the Download page or the dedicated article in the Developer Manual address this issue... If not, how would you rephrase those explanations to make things more clear?

...however, i wasn't able to see the protocol analyzer plugin working

Most likely, something is wrong with the implementation of your protocol analyzer. You didn't describe which protocol you try to parse, how exactly you do that, and which data you feed to the analyzer.

If you upload the sources of your protocol analyzer somewhere, we can take a look.

BTW, the tutorial has a link to an archive with a working analyzer of a made-up protocol. There are also real-life Modbus Analyzer and DF1 Analyzer; all those can be used as a reference when developing one of your own.

In any case, i was able to create a new protocol analyzer according to the help provided in - https://ioninja.com/doc/developer-manual/tutorial-plugin-analyzer.html
I've added the plugin to IO Ninja (using the settings dialog), and add the new plugin as a layer on top of 'Serial Tap'.
We are using a IO Ninja serial Tap device to sniffer serial packets (we use TTL).
I'm able to see the packet flow when the serial tap is connected, however, i wasn't able to see the protocol analyzer plugin working... the plugin parses the incoming data, and when a full packet is completed is sends a formatted string to the logger (just as done in the tutorial).
Is there something i'm missing ?
Many thanks.

IO Ninja features open-source Modbus and DF1 analyzers that could be used as a reference for approaching this task. Also, there's a tutorial in the Developer's Manual:

https://ioninja.com/doc/developer-manual/tutorial-plugin-analyzer.html

Is it possible to create such a template that can display different data structure according to the packet type

Just to ensure we're on the same page -- packet templates are for editing packets before transmission, not for displaying them in the log. After defining a packet template, you can conveniently prepare it with a property grid before transmission.

If that's what you try to do, then the answer is YES, you can define a packet template with a variable-sized payload length and a checksum in the end:

import "crc16.jnc"

pragma(Alignment, 1)

struct FE_BE_Packet {
	uint8_t StartOfPacket;
	uint16_t Length;
	uint8_t MessageID;
	uint16_t GlobalSequenceNumber;
	uint16_t MessageSequenceNumber;

	[ 	
		userAction = "Update checksum",
		autorun = "Auto-update checksum" 
	]
	void updateChecksum() {
		size_t size = dynamic sizeof(this) - sizeof(uint16_t); // without checksum
		uint16_t* checksum = (uint16_t*)((char*)this + size);
		*checksum = crc16_ansi(this, size, -1); // Modbus CRC
	}
}

Here is a quick rundown of what's happening here.

The structure definition itself is for the fixed-size header -- the header fields can be edited with a property grid. Beyond this header, you can enter a variable-sized payload in the hex editor.

Now, to the interesting part. We want to auto-update the checksum every time the user changes any fields in the header (or the payload itself). To achieve that, I create a method called updateChecksum(), mark it as userAction (so it's user-accessible via a hyperlink) and autorun (execute on every data change) using function attributes [...], and inside this method, I calculate a checksum and write it to the last two bytes of the packet. In my example, I use the Modbus checksum (CRC16-ANSI with the seed of 0xffff), but feel free to update it according to your needs.

Hope this helps!

Here is how out packet looks like:
pragma(Alignment, 1)

struct FE_BE_Packet
{
uint8_t StartOfPacket;
uint16_t Length;
uint8_t MessageID;
uint16_t GlobalSequenceNumber;
uint16_t MessageSequenceNumber;
char MessageData[70];
uint16_t CheckSumData;
}
The 'MessageData' is the one that keeps changes (size and content)

The new syntax is:

pragma(Alignment, 1)

For an explanation for jumping bytes and how packing and alignment affect the memory layout, please refer to this Wikipedia article: https://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C_structs_on_x86