A new free debugging tool open sourced this week aims to make life easier for security researchers trying to debug complex multiprocess applications — specifically, those trying to tackle the “nightmare” of tracing wildly heterogeneous inter-process communications (IPC) on Linux.
ipcdump was open sourced this week by Tel Aviv-based security firm Guardicore and covers IPC mechanisms “including pipes, fifos, signals, Unix sockets, loopback-based networking, and pseudoterminals”.
As the Guardicore team notes: “Debugging is difficult for single-process applications; for multiprocess ones, it’s a nightmare.
” Typically it is a juggling act of two instances of gdb , each with some byzantine follow-fork-mode configuration which tries to track only the right PIDs. One often needs to look through strace output for all the variations of recv (recvmsg? recvmmsg? recvfrom? Is that all of them?) just to understand who is interacting with whom. To make matters worse, there are lots of different types of IPC on Linux, and they don’t have common code flows, for the most part. A Unix socket and a FIFO both transfer bytes from one end to another, but that’s pretty much all they have in common.”
The tool was born out of some of the challenges Guardicore’s own team faced and can be used for everything from catching Chrome events to seeing how domain names are resolved on an Ubuntu machine. ipcdump is built of a series of collectors, Guardicore notes, each of which is in charge of a particular type of IPC event. For example,
- Loopback TCP and UDP
- Unix streams and datagrams
- Pseudo Terminals (pty)
As the team notes: “One of the key points for all of these is ipcdump’s ability to name the processes at either end of an IPC event. While sniffing over the loopback with tcpdump is a terrific way to understand the traffic you’re seeing between any two processes, it doesn’t actually tell you who those processes are. While you can certainly check port numbers against netstat, short-lived processes will wreak havoc on your ability to map out who really sent out each packet. Short-lived processes are one of the best use-cases for ipcdump in general; a lot of the bookkeeping it does internally is for tracking their creation and destruction.
Check it out and take it for a spin here.
Note: While we’re not prone to dishing out free promotion for vendors at The Stack, we’re typically happy to make an exception when it comes to spreading the word about useful, free, and well packaged OSS tools, of which Guardicore has its fair share — see also attack and breach simulation tool Infection Monkey. If you have one to share, tell us all about it).