Log Regex Markup

Highlight and Divide Data Using Regular Expressions

Purchase

All-in-one subscription

Individual capabilities

---

Required:

com.ioninja.log-regex-markup

$25

Add to cart

The Log Regex Markup engine is a unique and highly practical feature of IO Ninja, designed to enhance data log analysis with intuitive visual aids.

This engine uses user-defined regular expressions to automatically—and instantly!—highlight data patterns or insert packet delimiters. It’s powered by Google RE2, one of the world’s fastest and most reliable regular expression libraries.

How Does Regex Markup Improve Data Analysis?

Colorize Key Data

The simplest way to leverage the regex markup feature is by defining regular expressions for important tokens in your protocol and assigning them distinct colors for easy identification.

For example, as shown in the screenshot, all XTERM CSI escape sequences are automatically highlighted in pink.

Packetize Your Logs

Another powerful use case is defining packet boundaries by inserting delimiters with regular expressions.

For instance:

• If packets in your protocol begin with a specific header or preamble, you can write a regex to match those headers and insert delimiters before each match.
• Similarly, if your packets end with clear terminators, you can create a regex to match those terminators and insert delimiters after each match.

Handle Multiple Rules

Need to highlight different entities with different colors while simultaneously marking packet boundaries? No problem! With IO Ninja, you can define multiple rules and assign as many colors as needed to annotate everything you want to focus on.

The best part? It’s just as fast as applying a single regex!

Regex Engine Details

The IO Ninja regex engine is DFA-based, unlike the backtracking engines used in PCRE, Python, Ruby, and similar libraries.

While backtracking engines can suffer significant performance drops (sometimes exponentially!) depending on the complexity of the regex and the number of patterns, IO Ninja avoids these pitfalls. No matter how many markup rules or how complex the regular expressions are, the log will always be colorized and packetized FAST.

However, this design choice does come with some trade-offs. IO Ninja doesn’t support certain advanced regex features found in backtracking engines, such as backreferences and named groups.

Syntax

Since the regex markup engine is powered by Google RE2, it supports all the features of RE2. For a full syntax specification, refer to the official RE2 wiki page. If you’re looking for a quick start, check out the quick cheat sheet below—it’s more than enough to get you going.

Gallery