Log Regex Markup

Requires:

com.ioninja.log-regex-markup

The Log Regex Markup engine is yet another unique and highly useful feature of IO Ninja dedicated to providing visual aids when you analyze captured data logs.

The engine relies on user-defined regular expressions to automatically — and instantaneously! — highlight data patterns or insert packet delimiters. Under the hood, it is powered by Google RE2 — one of the world's fastest and well-tested regular expression libraries.

Colorize

The most straightforward way to use this regex markup feature is to write a regular expression for important tokens in your protocol — and then colorize (i.e., highlight those tokens with color).

For example, in the screenshot, all XTERM CSI escape sequences are automatically colored pink.

Packetize

An alternative approach is to insert delimiters between packet boundaries and use a regular expression to define such boundaries.

For instance, if packets in your protocol always start with a specific header/preamble, you can define a regular expression to match those headers and insert delimiters before matches.

Likewise, if packets have well-defined terminators — define a terminator-matching regex and insert delimiters after matches.

Multiple Rules

Want to colorize different entities with different colors? How about also highlighting packet boundaries at the same time? No problem! You can define as many rules and use as many colors as necessary to mark up everything you need.

And the best part — it's going to be just as fast as marking up with a single regex!

Regex Engine Specifics

The IO Ninja regular expression engine is a DFA-based one (unlike backtracking engines used in PCRE, Python, Ruby, etc.)

Backtracking engines could suffer from performance drops (sometimes exponential!) depending on particular regular expressions and the number of patterns. IO Ninja doesn't have these performance drawbacks — no matter what the regular expressions are and how many markup rules you define, the log will always get colorized and packetized FAST!

On the other hand, IO Ninja doesn't support some advanced regular expression features available in PCRE and other backtrackers — most notably, backreferences and named groups.

Syntax

Since the backend for our regex markup engine is based on Google RE2, we support everything it does. Please refer to the official RE2 wiki page for a complete syntax specification. Or check the quick cheat-sheet below — that's more than enough to get you started.

Gallery