Trouble capturing traffic using the Serial Monitor

I'm trying to use the evaluation copy to monitor serial activity on /dev/ttyUSB0. I got the kernel module installed and ran the app and created a serial monitor session on /dev/ttyUSB0. I get the "Capture started on /dev/ttyUSB0" message when I start capture but I don't see any data displayed. I know my device is sending data because the LEDs on the USB to serial adapter are flashing. Also, my application that is using the serial port is working. Why wouldn't I see the captured data when I start capturing?

I'm running on a Raspberry Pi 4.

@david-betz

Hello David,

Which version of tdevmon and which Raspbian (most importantly, the Linux kernel version) you are using?

$ ./tdevmon --version
$ screenfetch
$ # or uname -a

When you run the "Serial Monitor" plugin, do you see notifications about port open/close (but no reads/writes)? Or no notifications about port activity at all?

I get this far:

Session started
Capture started on /dev/ttyUSB0

pi@raspberrypi:~ $ sudo tdevmon --version
Tibbo Device Monitor v3.3.9 (arm32)
Built on: Sep 6 2021 09:37:19

Tibbo Device Monitor kernel module (arm32) v3.3.9
Built on: Jun 3 2022 16:40:26

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.15.32-v7l+ #1538 SMP Thu Mar 31 19:39:41 BST 2022 armv7l GNU/Linux

Does it initially crash?

Please reboot your Raspberry Pi, build and load tdevmon.ko, then, instead of running IO Ninja, navigate to tdevmon-3.3.9-linux-arm32/bin and run:

$ ./tdevmon -m /dev/ttyUSB0

Will it segfault?

Please try replacing the kernel module sources in share/tdevmon/src/lkm with these:

lkm-src.tar.xz

It looks like there's a bug in page protection removal code on ARM introduced in tdevmon-3.3.9...

@vladimir Yes, this does give me a segfault. This is even after I installed the new kernel module you posted in your other message.

Here is the output of dmesg after the segfault.

[ 91.878302] tdevmon: hooking /dev/ttyUSB0 (fops: 9174bd4c)
[ 91.878332] 8<--- cut here ---
[ 91.878347] Unable to handle kernel paging request at virtual address c0c48068
[ 91.878363] pgd = 2f6d8e2b
[ 91.878378] [c0c48068] *pgd=80000000007003, *pmd=40000000c0079d(bad)
[ 91.878421] Internal error: Oops: a0e [#1] SMP ARM
[ 91.878433] Modules linked in: exfat rfcomm cmac algif_hash aes_arm_bs crypto_simd cryptd algif_skcipher af_alg bnep hci_uart btbcm bluetooth ecdh_generic ecc snd_soc_hdmi_codec 8021q garp stp llc joydev ftdi_sio usbserial brcmfmac brcmutil v3d cfg80211 gpu_sched rfkill raspberrypi_hwmon i2c_brcmstb bcm2835_isp(C) bcm2835_v4l2(C) bcm2835_codec(C) snd_bcm2835(C) bcm2835_mmal_vchiq(C) v4l2_mem2mem videobuf2_vmalloc videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc vc_sm_cma(C) vc4 rpivid_mem cec drm_kms_helper snd_soc_core snd_compress snd_pcm_dmaengine snd_pcm snd_timer snd syscopyarea sysfillrect sysimgblt fb_sys_fops nvmem_rmem uio_pdrv_genirq uio sg tdevmon(O) i2c_dev drm fuse drm_panel_orientation_quirks backlight ip_tables x_tables ipv6
[ 91.878787] CPU: 3 PID: 1116 Comm: tdevmon Tainted: G C O 5.15.32-v7l+ #1538
[ 91.878801] Hardware name: BCM2711
[ 91.878810] PC is at Hook_create+0x150/0x230 [tdevmon]
[ 91.878854] LR is at 0x0
[ 91.878863] pc : [<bf192e74>] lr : [<00000000>] psr: 40000013
[ 91.878873] sp : c2b03e70 ip : 00000020 fp : c2b03ea4
[ 91.878883] r10: 00000000 r9 : bf19a448 r8 : c341bcc8
[ 91.878891] r7 : c2b03eac r6 : c0c48030 r5 : c0c480b0 r4 : c3534180
[ 91.878901] r3 : bf1935f8 r2 : 00000000 r1 : c0c480b0 r0 : c353418c
[ 91.878910] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 91.878924] Control: 30c5383d Table: 04a73b00 DAC: 55555555
[ 91.878932] Register r0 information: slab kmalloc-192 start c3534180 pointer offset 12 size 192
[ 91.878967] Register r1 information: non-slab/vmalloc memory
[ 91.878982] Register r2 information: NULL pointer
[ 91.878996] Register r3 information: 13-page vmalloc region starting at 0xbf191000 allocated at load_module+0xb94/0x2840
[ 91.879025] Register r4 information: slab kmalloc-192 start c3534180 pointer offset 0 size 192
[ 91.879057] Register r5 information: non-slab/vmalloc memory
[ 91.879070] Register r6 information: non-slab/vmalloc memory
[ 91.879082] Register r7 information: non-slab/vmalloc memory
[ 91.879095] Register r8 information: slab filp start c341bcc0 pointer offset 8
[ 91.879122] Register r9 information: 13-page vmalloc region starting at 0xbf191000 allocated at load_module+0xb94/0x2840
[ 91.879141] Register r10 information: NULL pointer
[ 91.879154] Register r11 information: non-slab/vmalloc memory
[ 91.879167] Register r12 information: non-paged memory
[ 91.879179] Process tdevmon (pid: 1116, stack limit = 0x3d367e2b)
[ 91.879190] Stack: (0xc2b03e70 to 0xc2b04000)
[ 91.879202] 3e60: 00000000 00c0079d 04400000 3391a86e
[ 91.879215] 3e80: c2b03ea4 c341bcc0 bf19a440 c341b300 c0c48030 00000000 c2b03ee4 c2b03ea8
[ 91.879226] 3ea0: bf191f04 bf192d30 c2b03ee4 00000000 bf196e98 3391a86e c2b03f34 40086d0b
[ 91.879238] 3ec0: c341b300 c4a73cc0 00000000 c341b300 be942270 00000003 c2b03f0c c2b03ee8
[ 91.879249] 3ee0: bf1929d4 bf191d50 be942270 3391a86e 40086d0b c1205048 0000000c 00000000
[ 91.879261] 3f00: c2b03fa4 c2b03f10 c046ca40 bf1928a8 ffffff9c 00000000 00000000 0005a094
[ 91.879272] 3f20: c2b02000 00000142 c2b03f94 c341b300 c045333c c04516f8 00000902 00000000
[ 91.879283] 3f40: 00000000 00000000 00000000 00000000 00000902 00000000 00000000 00000000
[ 91.879294] 3f60: 00000000 00000000 c2b03fac 3391a86e c02152e0 3391a86e b6f13500 be942270
[ 91.879306] 3f80: 0006f000 0000000c 00000036 c0200244 c2b02000 00000036 00000000 c2b03fa8
[ 91.879317] 3fa0: c0200040 c046c920 be942270 0006f000 00000003 40086d0b be942270 00000015
[ 91.879328] 3fc0: be942270 0006f000 0000000c 00000036 be942258 be9423b4 40086d0b 00000015
[ 91.879339] 3fe0: b6d8d1a0 be942224 00026298 b6d8d1ac 60000010 00000003 00000000 00000000
[ 91.879347] Backtrace:
[ 91.879359] [<bf192d24>] (Hook_create [tdevmon]) from [<bf191f04>] (Device_p_connectImpl+0x1c0/0x274 [tdevmon])
[ 91.879420] r8:00000000 r7:c0c48030 r6:c341b300 r5:bf19a440 r4:c341bcc0
[ 91.879428] [<bf191d44>] (Device_p_connectImpl [tdevmon]) from [<bf1929d4>] (Device_fop_ioctl+0x138/0x3f0 [tdevmon])
[ 91.879484] r10:00000003 r9:be942270 r8:c341b300 r7:00000000 r6:c4a73cc0 r5:c341b300
[ 91.879493] r4:40086d0b
[ 91.879501] [<bf19289c>] (Device_fop_ioctl [tdevmon]) from [<c046ca40>] (sys_ioctl+0x12c/0xa20)
[ 91.879539] r7:00000000 r6:0000000c r5:c1205048 r4:40086d0b
[ 91.879547] [<c046c914>] (sys_ioctl) from [<c0200040>] (ret_fast_syscall+0x0/0x1c)
[ 91.879564] Exception stack(0xc2b03fa8 to 0xc2b03ff0)
[ 91.879575] 3fa0: be942270 0006f000 00000003 40086d0b be942270 00000015
[ 91.879587] 3fc0: be942270 0006f000 0000000c 00000036 be942258 be9423b4 40086d0b 00000015
[ 91.879597] 3fe0: b6d8d1a0 be942224 00026298 b6d8d1ac
[ 91.879609] r10:00000036 r9:c2b02000 r8:c0200244 r7:00000036 r6:0000000c r5:0006f000
[ 91.879617] r4:be942270
[ 91.879629] Code: e284000c eb57f4d4 e30335f8 e34b3f19 (e5863038)
[ 91.879641] ---[ end trace a838db62019b3702 ]---
[ 326.926021] v3d fec00000.v3d: MMU error from client CLE (4) at 0x74a1000, pte invalid

OK, thanks. I will try to match your Raspbian as close as possible and see if I get it.

@vladimir Thanks!

Hmmm.. I installed exactly the same kernel as on your box:

vladimir@raspberrypi:~ $ uname -a
Linux raspberrypi 5.15.32-v7+ #1538 SMP Thu Mar 31 19:38:48 BST 2022 armv7l GNU/Linux

Then overwritten the contents of src/lkm with the archive I uploaded earlier. It works!

vladimir@raspberrypi:~/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm $ make
make -C /lib/modules/5.15.32-v7+/build/ M=/home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm modules
make[1]: Entering directory '/usr/src/linux-headers-5.15.32-v7+'
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/module.o
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Device.o
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Hook.o
/home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Hook.c: In function ‘Hook_stop’:
/home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Hook.c:143:22: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
  143 |   self->m_fops->read && self->m_fops->read != Hook_fop_read ||
      |   ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Hook.c:144:23: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
  144 |   self->m_fops->write && self->m_fops->write != Hook_fop_write ||
      |   ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Hook.c:145:27: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
  145 |   self->m_fops->read_iter && self->m_fops->read_iter != Hook_fop_read_iter ||
      |   ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Hook.c:146:28: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
  146 |   self->m_fops->write_iter && self->m_fops->write_iter != Hook_fop_write_iter ||
      |   ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Hook.c:147:32: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
  147 |   self->m_fops->unlocked_ioctl && self->m_fops->unlocked_ioctl != Hook_fop_unlocked_ioctl ||
      |   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Hook.c:148:30: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
  148 |   self->m_fops->compat_ioctl && self->m_fops->compat_ioctl != Hook_fop_compat_ioctl
      |   ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Connection.o
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/HashTable.o
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/ScatterGather.o
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/FileNameFilter.o
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/lkmUtils.o
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/stringUtils.o
  LD [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/tdevmon.o
  MODPOST /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/Module.symvers
  CC [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/tdevmon.mod.o
  LD [M]  /home/vladimir/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm/tdevmon.ko
make[1]: Leaving directory '/usr/src/linux-headers-5.15.32-v7+'

vladimir@raspberrypi:~/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm $ sudo insmod tdevmon.ko
vladimir@raspberrypi:~/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/share/tdevmon/src/lkm $ 
vladimir@raspberrypi:~/Projects/ioninja/devmon/build/make-arm32/tdevmon-3.3.9-a-linux-arm32/bin $ sudo ./tdevmon -m /dev/ttyUSB0
Monitoring /dev/ttyUSB0 (module: usbserial), hit Ctrl+C to interrupt...

16:13:19.821 open
    pid:             2604
    tid:             2604
    result:          0
    file id:         00000000b902eb40
    flags:           00020241
    mode:            00000002
    file name:       /dev/ttyUSB0

16:13:19.821 write_iter
    pid:             2604
    tid:             2604
    result:          4
    file id:         00000000b902eb40
    offset:          0
    buffer size:     4
    data size:       4

    0000  61 62 63 0a                                      abc.

16:13:19.822 close
    pid:             2604
    tid:             2604
    result:          0
    file id:         00000000b902eb40

Are you sure you rebooted, or at the very least, removed the previous instance of tdevmon.ko via rmmod before testing? Also, please make sure you overwrite all source files, not simply unpack the archive into src/lkm (thus creating src/lkm/lkm).

Actually, to make things more straightforward, here's a re-packaged patch:

tdevmon-3.3.9-a-linux-arm32.tar.gz

Please let me know if it works (again, reboot before testing).