add traffic recording to pfsense easily

First of all, if you might ask yourself the question “Why should I?” here are two answers - you choose which one works best for you:
1) Because you can.
2) In case of an incident it might be very helpful and there is no traveling back in time and get a second chance.

It might not be the best idea to save the traffic to the actual file system your pfsense is running on. In the past the file system was mounted read-only (that changed recently) but there are more reasons to that. It is simply not very handy. If you encounter an incident and you have to copy all the traffic over the network… might take too mutch time, the network might even no longer be available. But again: You decide.

I decided to use a external usb storage attached to the pfsense box. In case I needed the data, I can simply walk over, grep the usb stick/external hd and attach it to the analysis system. No copy, no impact on the running pfsense system - sneakernet for the win.

For the sake of testing I used one of my usb sticks floating around at my desk. It only has 8 GB and is not very fast, so definitely not enough for a real network. You should be able to adopt to any larger size after reading the rest of the post.

First thing to ensure is, we can run commands at boot time on our pfsense. Most simple way I found was to install the package “Shellcmd” via pfsense buildin package manager. If you read the description “The shellcmd utility is used to manage commands on system startup.” this is pretty much what we are going to do.

Next thing to do: compiling a small script we can use to call at startup

#!/bin/sh
#
# Startup script for trdump via tcpdump
#
# description: trdump control script
# processname: tcpdump


# -size 100  count 72 capture 72 files of 100 MB; this will fit the 8 GB stick
PCAP=/mnt/netrec/trdump.pcap
SIZE=100
COUNT=72
INTERFACE=re0
PIDFILE=/var/run/tcpdump

start() {
        if [ -f $PIDFILE ]; then
                echo "PID File $PIDFILE exists"
                exit 1
        fi

  if (/sbin/mount | /usr/bin/grep -q /mnt/netrec ) then
      /usr/bin/logger "/mnt/netrec already mounted"
  else
      /sbin/mount -t msdosfs /dev/da1s1 /mnt/netrec/
      if($?) then
          # mount successfull
          /usr/bin/logger "/dev/da1s1 mounted to /mnt/netrec"
      else
          # mount not successfull
          /usr/bin/logger "mounting /dev/da1s1 to /mnt/netrec failed exiting"
          exit
     fi
  fi



  /usr/bin/logger "starting traffic dump"
  # if we reach the code here, our disk is mounted
  # start recording
  # -s 0 collect entire packet contents
  # -n Don't convert addresses (i.e., host addresses, port numbers, etc.) to names
  # -C 100 -W 72 capture 72 files of 100 MB; this will fit the 8 GB stick
  # filter out the ip of the the tor node
  /usr/sbin/tcpdump -i $INTERFACE -n -s 0 -C $SIZE -W $COUNT -w $PCAP 'not (src host 78.142.145.141 or dst 78.142.145.141)' >/dev/null 2>&1 &


        echo $! > $PIDFILE
        exit 0
}

stop() {
        if [ ! -f $PIDFILE ]; then
                echo "PID File $PIDFILE does not exist"
                exit 1
        fi

      /usr/bin/logger "stoping traffic dump"
        kill -HUP `cat $PIDFILE` && rm $PIDFILE
        exit $@
}

status() {
        if [ ! -f $PIDFILE ]; then
                echo "PID File $PIDFILE does not exist"
                exit 0
        fi
        ps -fp `cat $PIDFILE`
        exit 0
}

case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  status)
        status
        ;;
  *)
        echo "Usage: $0 {start|stop|status}"
        exit 1
esac


exit

This script will do some logging, it will try to mount the usb stick (not the most fail-safe way, but good enough for now) and as a little extra it offers a start, stop and status command.
If you what to use the script you should be fine after adopting the parameter section in the beginning of the script, the mount command at line 25 and changing the tcpdump filter at line 45. I use this filter to not dump the traffic of my Tor node. Just removing everything between the single quotes and the single quotes itself should be fine.

Installing the “Shellcmd” package will add a menu item called “Shellcmd” in the “Services” menu. This is were the configuration is done to start the dump script if pfsense boots. By hitting the “add” button one can configure a new task. Put in the path to the script followed by a whitespace and the parameter “start”. See example below.

/usr/local/trdump.sh start

As Shellcmd Type just leave “shellcmd” and add a meaningful description in the corresponding field. Hit save and do not forget to make the script executable and attach you usb storage.
Test the script if it runs without errors by manually calling it and providing “start” as a parameter. If it works you can let it run or stop it and reboot your pfsense box and verify if it works on a reboot as well.

At the end a few words on the tcpdump flags:
-C $SIZE -W $COUNT
With SIZE equals 100 and COUNT equals 72, tcpdump captures 72 files of 100 MB. If 72 files exist, tcpdump will roll over and dump to the first file again.
This ensures that the storage (in this case the 8 GB usb stick) does not run out of space. It make sence to save the dump to smaller chunks and not one or two big files. If you do not believe me: dump a few GB of traffic and try to open it with wireshark for example.

Feel free to use the script etc. to build your own custom script or use it and adopt what is needed for your environment.

less ./b00010111/blog

Add Traffic Recording to Pfsense Easily