Save log data as individual binary files

Hello! I am a very new IO Ninja user so I'm not sure of all the functionality yet so I apologies if this is something that is easily done.

I am a security researcher and I'm using IO Ninja to investigate how applications are using named pipes. I would like to extract all the data sent to a named pipe from IO Ninja but I want to save the binary data into individual files that I can replay using other code I have written.

I found that I save the data as txt but it's all the logs instead of individual ones. Is there any functionality that would be able to do with?

Hello Dan,

If you want to filter data flowing in one direction (i.e., leave TX-only or RX-only), you can apply the TX/RX Filter.

Beware, that with the Pipe Monitor plugin, each block of data will be visible twice (i.e., once as TX for one side of the connection, and once as RX for the other side). If you want to get rid of this duplication, you can first filter the log using the file ID (i.e., first, leave only data visible by one side of the connection) -- and then apply the TX/RX filter to leave TX-only or RX-only.

And of course, you can post-process the captured .njlog log file in any custom way you want -- with a script written in any language of your choice. The log file format is very simple and all relevant definitions are open-source:

scripts/api/log_RecordFile.jnc
scripts/api/log_RecordCode.jnc

Thanks for this information! Can you provide the full urls to the log_RecordFile.jnc and log_RecordCode.jnc? I couldn’t find them with google search.

@vladimir said in Save log data as individual binary files:

Hello Dan,

If you want to filter data flowing in one direction (i.e., leave TX-only or RX-only), you can apply the TX/RX Filter.

Beware, that with the Pipe Monitor plugin, each block of data will be visible twice (i.e., once as TX for one side of the connection, and once as RX for the other side). If you want to get rid of this duplication, you can first filter the log using the file ID (i.e., first, leave only data visible by one side of the connection) -- and then apply the TX/RX filter to leave TX-only or RX-only.

And of course, you can post-process the captured .njlog log file in any custom way you want -- with a script written in any language of your choice. The log file format is very simple and all relevant definitions are open-source:

scripts/api/log_RecordFile.jnc
scripts/api/ok

The scripts folder is a part of every installation package (i.e., <ioninja-root-dir>/scripts/).

The scripting language is Jancy, but it's a form of a safe scripting dialect of C/C++, so you should have no problems reading it.

I think it would be a very helpful feature to add to IO ninja to just be able to save a log as a raw binary file for each direction. I often have serial port logs that I just want to save to a bin file as-is, and I end up closing IO ninja and just going back to something like CoolTerm which supports this natively.

(it also supports recording directly to a file, in case the computer crashes or powers off during logging)

I think it would be a very helpful feature to add to IO ninja to just be able to save a log as a raw binary file for each direction.

It's very easy to achieve with a log filter/observer plugin script.

However, I can't say I find it very useful to have two raw binary files containing just the raw bytes flowing in each directing -- how would you reconstruct the mapping between requests and replies in case of some non-trivial conversation between the client and server?

I often have serial port logs that I just want to save to a bin file as-is, and I end up closing IO ninja and just going back to something like CoolTerm which supports this natively.

If it's a uni-directional log kind of thing (e.g., your device keeps dumping debug printfs to a RS232, and you want to save it to a file) -- this can be done using Session Linking. Open a Serial session, open a File Stream session in Write-only mode and point it to some file, then "link" those sessions. Everything that the Serial session reads from the device, will be redirected to your file.

it also supports recording directly to a file, in case the computer crashes or powers off during logging

Actually, IO Ninja always keeps its logs in disk files. When you "Save Log", the temporary .njlog file simply gets copied into the file you select. But if you "Save Session", those .njlog files will get stored in the specified folder, and you will know exactly where to find them in case of a crash.

But once again, a dedicated plugin that would explicitly save RX to one file and TX to another is quite easy to implement. I don't know, maybe we should add it to one of the upcoming releases as an official plugin (or a sample)?