Categories

Graphing snort.stats with gnuplot

So we have been trying to create some visual graphs for all of our Snort instances so we can easily tell when traffic/processor/alert spikes are occurring. This lead us to the very useful snort.stats file. However, if you are compiling Snort (or using someone elses packages) you need to ensure that it was compiled/configured with the --enable-perfprofiling option. This allows you to create the snort.stats file be configuring the preprocessor inside of your snort.conf file:

# Just an example, check the Snort docs to see what works best for you:
# You *must* set the full/path filename for this output
preprocessor perfmonitor: time 60 file /var/log/snort/snort.stats pktcnt 100000

The docs on the contents of the snort.stats file seem to be a bit out of date, and even the comments in the source code seem to be a bit lacking (I have no clue what the last seven columns are for). But here they are for refrence purposes straight out of the comments section of snort-2.8.4/src/preprocessors/perf-base.c:

 1 - unixtime (in secs since epoch)
 2 - %pkts dropped
 3 - mbits/sec (wire)
 4 - alerts/sec
 5 - K-Packets/Sec (wire)
 6 - Avg Bytes/Pkt  (wire)
 7 - %bytes pattern matched
 8 - syns/sec
 9 - synacks/sec
10 - new-sessions/sec (tcp stream cache)
11 - del-sessions/sec (tcp stream cache)
12 - total-sessions open (tcp stream cache)
13 - max-sessions, lifetime (tcp stream cache)
14 - streamflushes/sec
15 - streamfaults/sec
16 - streamtimeouts
17 - fragcreates/sec
18 - fragcompletes/sec
19 - fraginserts/sec
20 - fragdeletes/sec
21 - fragflushes/sec
22 - current-frags open (frag cache)
23 - max-frags (frag cache)
24 - fragtimeouts
25 - fragfaults
26 - num cpus (following triple is repeated for each CPU)
27 - %user-cpu usage
28 - %sys-cpu usage
29 - %idle-cpu usage
30 - mbits/sec (wire)
31 - mbits/sec (ip fragmented)
32 - mbits/sec (ip reassembled)
33 - mbits/sec (tcp stream rebuilt)
34 - mbits/sec (app layer)
35 - Avg Bytes/Pkt  (wire)
36 - Avg Bytes/Pkt  (ip fragmented)
37 - Avg Bytes/Pkt  (ip reassembled)
38 - Avg Bytes/Pkt  (tcp stream rebuilt)
39 - Avg Bytes/Pkt  (app layer)
40 - K-Packets/Sec (wire)
41 - K-Packets/Sec (ip fragmented)
42 - K-Packets/Sec (ip reassembled)
43 - K-Packets/Sec (tcp stream rebuilt)
44 - K-Packets/Sec (app layer)
45 - Pkts recieved
46 - Pkts dropped
47 - Blocked-KPackets  (wire)
48 - udp-sessions
49 - max-udp-sessions
50 - del-udp-sessions/sec (udp stream cache)
51 - new-udp-sessions/sec (udp stream cache)
52 - max-sessions, interval (tcp stream cache)
53 - curr-tcp-sessions-initializing (tcp stream cache, of total-sessions open)
54 - curr-tcp-sessions-established (tcp stream cache, of total-sessions open)
55 - curr-tcp-sessions-closing (tcp stream cache, of total-sessions open)
56 - tcp-sessions-mistream/sec (tcp stream cache, of new-sessions/sec)
57 - tcp-sessions-closed/sec (tcp stream cache, of del-sessions/sec)
58 - tcp-sessions-timedout/sec (tcp stream cache, of del-sessions/sec)
59 - tcp-sessions-pruned/sec (tcp stream cache, of del-sessions/sec)
60 - tcp-sessions-dropped_async/sec (tcp stream cache, of del-sessions/sec)
61 - hosts in attribute table
62 - attribute table reloads
63 - ???
64 - ???
65 - ???
66 - ???
67 - ???
68 - ???
69 - BLANK???

Using that list as a refrence, we can create scripts to use GNUplot to make some graphical output we can put on a webpage for monitoring. You could also use RRDTool, but after trying to read about how to create databases specific for the use of it for about ten minutes, I moved on to the easier to use GNUplot solution. Here is a little script that creates some nice graph for CPU utilization:

#!/bin/bash
tail -n 60 /var/log/snort/snort.stats > /tmp/snort.stats.tmp

tail -n 60 will grab the last 60 lines of the file which for our configuration will graph the last hours worth of data. Since we are creating a log entry every minute (remember the snort.conf entry?: preprocessor perfmonitor: time 60) then the perfmonitor Snort preprocessor will create an entry every sixty seconds.

cd /tmp
gnuplot
set datafile separator ","

Let GNUplot know that the data is seperated by commas (snort.stats file is a simple CSV).

set terminal png
set output 'cpu.png'
set xlabel "Time"
set timefmt "%s"

set timefmt to %s lets GNUplot know that the time is stored in epoch/UNIX time (seconds since 1/1/1970 00:00).

set xdata time
set ylabel "CPU Utilization (%)"
set yrange [0:100]

set yrange [0:100] is set because hopefully you aren’t using more than 100% processor utilization. ;)

plot 'snort.stats.tmp' using 1:27 with lines title "User %", 'snort.stats.tmp' using 1:28 with lines title "System %", 'snort.stats.tmp' using 1:29 with lines title "Idle %"
exit

Here we are using the plot command to plot the 1st column (1 – unixtime) against the values of the 27th, 28th and 29th columns (27 – %user-cpu usage, 28 – %sys-cpu usage, 29 – %idle-cpu usage).

mv /tmp/cpu.png /var/www/cpu.png
rm /tmp/snort.stats.tmp

Without comments:

#!/bin/sh
tail -n 60 /var/log/snort/snort.stats > /tmp/snort.stats.tmp
cd /tmp
gnuplot
set datafile separator ","
set terminal png
set output 'cpu.png'
set xlabel "Time"
set timefmt "%s"
set xdata time
set ylabel "CPU Utilization (%)"
set yrange [0:100]
plot 'snort.stats.tmp' using 1:27 with lines title "User %", 'snort.stats.tmp' using 1:28 with lines title "System %", 'snort.stats.tmp' using 1:29 with lines title "Idle %"
exit
mv /tmp/cpu.png /var/www/cpu.png
rm /tmp/snort.stats.tmp

Resulting output will look something like this (this graph was made for more than just an hours worth of data):

cpu

Leave a Reply

 

 

 

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>