CLI use (ioninja - serial tap)

Hello.
I'm a happy owner of a (tibbo) serial tap and ioninja SW.
Now, I want to use them in order to capture data, in a remote location, unattended and with bad communications.
I am going to use a Rasperry (Arch Arm) with a GPRS modem (not sure if 3G is available).
I do not need the GUI (and it could be a problem if I have to use it!).
Ideally, I would like to be able to save a log, and download it from time to time. I've seen that there is an executable "ioninja-server" that seems that could do the trick, but I have not found detailed documentation (just the "-h", and a short reference in your web).

So the question are: Is there any way to do what I want? With this "ioninja-server"? how?

Thank you very much!

Apologies for the delayed reply... Still, better late than never

Alas, currently, there's no support for remote monitoring via Serial Taps -- fiddling with ioninja-server won't cut it... However, it's on our immediate TODO list; we are working on a major update scheduled for the beginning of 2022 which -- among many other things -- will allow controlling Serial Taps and other IO Ninja hardware sniffers connected to remote ARM-based Linux boxes (such as Raspberry Pis).

If waiting for the update is not an option, I suggest writing your own program which would read from a Serial Tap using libusb (or any other low-level USB framework available in the programming language of your choice) and dump the captured data into a file.

The low-level details of USB communication with a Serial Tap (USB VIDs/PIDs, IN/OUT endpoints, packet formats, etc) can be checked in the open-source implementation of the official Serial Tap plugin (see file scripts/plugins/SerialTap/SerialTapSession.jnc). If you are familiar with C-family languages (C, Java, JavaScript, etc) you should be able to read and understand pretty much everything. And of course, feel free to ask any implementation-related questions here.

Hope this clarifies the issue.

Hello. Do not worry about the delay, we are all very busy
It will be good to have it. In the meantime, libusb could be an option. I used to program in C -many time ago- and I will try (perhaps, I will eventually bother you a bit, with some questions).
Thank your! Regards!

Hello. Thank you! I've seen that the recent release includes a tool in order to achieve what I wanted to do.
Just a improvement for future releases, it would be great if the tool would save a log, in ".njlog" format, part of my problem is the lack of communications, so keep a ssh session alive it is not an option (I saw that it is possible save "pcap" for ethernet captures, but -obviously- we are talking about serial).
Also, I checked the "write to file" option, and I have two comments:

1. I saw that as "sidecar" of the sniffed data there are some other data (4 bytes ?), I suspect it is info about the data origen, perhaps used by the "ioninja" SW in order to identify it. Is there any place where we can check the meaning of these bytes?
2. I suspect that inside this "sidecar" data there is no timestamps. The HW probably does not timestamp its raw data... but it would be quite necessary that the output files writen by the "ioninja-hwc" do include timestamps.
   Regards!!

Hello Jose,

Once again, apologies for the delayed reply

Yes, in IO Ninja 5.1 we added a CLI tool (ioninja-hwc) for direct communication with Serial Tap and other IO Ninja hardware. The main purpose of this tool was to network-enable the IO Ninja Taps (i.e., to access those over SSH). As such, the output from this tool must satisfy certain requirements (e.g., reliably distinguish between ioninja-hwc messages and SSH server errors, like ioninja-hwc not found). The .njlog format is not very suitable for that, that's why we didn't use it by default (we actually used the protocol from our test fixture for Serial and other Taps).

This said, we can of course add .njlog output as an extra feature -- just like we have .pcap output. We probably will do that in one of the upcoming releases.

1. Regarding the format of the output -- since all the scripts in IO Ninja are open-source, you can check the sources of decoder in ioninja/scripts/common/io_HwcProto.jnc. This file contains file structure definitions, packet type code constants, and the decoder itself.

2. You are right, there are no timestamps in this protocol. Serial Tap doesn't timestamp individual bytes (Serial Tap basically is a dual-channel USB-to-UART). We can, of course, add timestamps inside ioninja-hwc, but in case of SSH communications, it's not that much different from timestamping on the SSH client side (as we do now). And after we support the .njlog format for file output, adding timestamps to the protocol most likely will not be necessary.

Let me know what you think!

Hello Vladimir,
No worry about the delay and thank you !
Ok! I think that I will be able to use the current output, and it could solve my problems (at least one of them).
Eventually, if you could add the function of save as ".njlog", it would be great.
As a piece of info, that could help someone, I am currently trying "xpra" (https://xpra.org/), it is a kind of X server that allows remote access. This allows run ioninja remotely (and keep it alive permanently) and "attach" me to it from time to time remotely.
Regards!
Josep

Yes, we plan to add the .njlog output soon, so that would solve many problems.

Running a remote X-server is an option, yes. It actually was the only option for running Serial Tap remotely before the release of ioninja-hwc. Now that we have ioninja-hwc, it should be the preferred approach in most cases.

We also consider extending the Serial over SSH (and other remote tap plugins) with an option of keeping the ioninja-hwc process persistent across SSH reconnects (via screen or some other terminal manager).