Well the good folks over at SecurixLive.com have already fixed this and a few other little things and rolled it into their latest release of Barnyard2 v1.8-beta1. Go and get it!
We have identified an issue with Barnyard2 version 1.7 build 255 that causes duplicate entries to be created in the Sguil database. The issue was that the entries in the unified2 output from Snort will be read into two seperate lists (AlertList and LogList) as the unified2 log file is processed. Elsewhere in the Barnyard2 code, all the relavent information is stored into a buffer for creating the Sguil database entry for the alert (summary of event) and the log (offending packet) prior to firing off the output logging. The debugging output shows you that the SPECIAL style output is going to be used for the event/packet combo that was pulled out of the unified2 output file:
spi_unified2.c:151: Header: Type=7 (52 bytes) spi_unified2.c:159: Reading record type=7 (52 bytes) spi_unified2.c:320: Type: Event ------------------------------------------- spi_unified2.c:322: sensor_id = 0 spi_unified2.c:324: event_id = 4 spi_unified2.c:326: event_second = 1260520646 spi_unified2.c:328: event_microsecond = 383469 spi_unified2.c:330: generator_id = 1 spi_unified2.c:332: signature_id = 2009236 spi_unified2.c:334: signature_revision = 5 spi_unified2.c:336: classification_id = 21 spi_unified2.c:338: priority_id = 1 spi_unified2.c:353: ip_source = 192.168.1.5 spi_unified2.c:356: sport_itype = 57019 spi_unified2.c:359: ip_destination = 71.191.147.210 spi_unified2.c:361: dport_icode = 80 spi_unified2.c:364: ip_protocol = 6 spi_unified2.c:366: packet_action = 0 spi_unified2.c:151: Header: Type=2 (244 bytes) spi_unified2.c:159: Reading record type=2 (244 bytes) spi_unified2.c:403: Type: Packet ------------------------------------------ spi_unified2.c:405: sensor_id = 0 spi_unified2.c:407: event_id = 4 spi_unified2.c:409: event_second = 1260520646 spi_unified2.c:411: linktype = 1 spi_unified2.c:413: packet_second = 1260520646 spi_unified2.c:415: packet_microsecond = 383469 spi_unified2.c:417: packet_length = 216 spi_unified2.c:422: packet = 18 01 bb 24 decode.c:113: Decoding linktype 1 decode.c:314: Packet! decode.c:314: caplen: 216 pktlen: 216 decode.c:341: 0:21:9B:69:E0:9 -> 0:18:1:BB:24:4F decode.c:345: type:0x800 len:0xD8 decode.c:355: IP datagram size calculated to be 202 bytes decode.c:2648: Packet! decode.c:2822: IP Checksum: OK decode.c:2899: IP header length: 20 decode.c:3023: TCP th_off is 5, passed len is 182 decode.c:3103: TCP Checksum: OK decode.c:3107: tcp header starts at: 0x80dcd9e spooler.c:655: Firing SPECIAL style (Packet+Event)
So the debugging shows us only firing off once with SPECIAL style (Packet+Event). But if we look at the output presented to STDOUT, we see that the same alert inserted twice into the Sguil server (we have omitted packet contents for brevity):
sguil: sending "RTEVENT 0 1 199 sguil 4 4 {2009-12-11 03:37:26} 1 2009236 5 {ET USER_AGENTS Pigeon.AYX/AVKill Related User-Agent (CTTBasic) } {2009-12-11 03:37:26} 1 trojan-activity 3232235781 192.168.1.5 1203737554 71.191.147.210 6 4 5 0 202 2472 2 0 128 21319 {} {} {} {} {} 57019 80 309364297 2715104879 5 0 24 16425 7600 0 {} {}"
sguil: Received: Confirm 199
sguil: sending "RTEVENT 0 1 200 sguil 4 4 {2009-12-11 03:37:26} 1 2009236 5 {ET USER_AGENTS Pigeon.AYX/AVKill Related User-Agent (CTTBasic) } {2009-12-11 03:37:26} 1 trojan-activity 3232235781 192.168.1.5 1203737554 71.191.147.210 6 4 5 0 202 2472 2 0 128 21319 {} {} {} {} {} 57019 80 309364297 2715104879 5 0 24 16425 7600 0 {} {}"
sguil: Received: Confirm 200
So this lead us to inspect src/spooler.c which lead us to src/plugbase.c CallOutputPlugins() function. While reviewing it, we notice that if the OUTPUT_TYPE__SPECIAL (which Sguil is) is set, then it will process idx-func() once for the entry in the AlertList and once for the LogList:
void CallOutputPlugins(OutputType out_type, Packet *packet, void *event, uint32_t event_type)
{
OutputFuncNode *idx = NULL;
if (out_type == OUTPUT_TYPE__SPECIAL)
{
idx = AlertList;
while (idx != NULL)
{
idx->func(packet, event, event_type, idx->arg);
idx = idx->next;
}
idx = LogList;
while (idx != NULL)
{
idx->func(packet, event, event_type, idx->arg);
idx = idx->next;
}
}
So if we remove the processing the second time around for the LogList, we are able to stop the double entries from being created in the database. Below is what the patch (which you can download here) looks like. Obviously, you can just delete the lines with the single minus sign in front of them that are located within the CallOutputPlugins() function of src/plugbase.c:
--- plugbase.c 2009-10-17 06:08:55.000000000 -0400
+++ plugbase.c.new 2009-12-11 21:59:22.000000000 -0500
@@ -543,13 +543,6 @@
idx->func(packet, event, event_type, idx->arg);
idx = idx->next;
}
-
- idx = LogList;
- while (idx != NULL)
- {
- idx->func(packet, event, event_type, idx->arg);
- idx = idx->next;
- }
}
else
{
To apply the patch, move it to the src/ directory within the Barnyard2 v 1.7 distribution directory and perform the following then compile/install as normal:
#patch plugbase.c < plugbase.c.patch
Additional things we discovered during the troubleshooting process were how to enable the debugging within Barnyard2. You must first compile it with the --enable-debug option, then prior to running it you must set the environment variable BARNYARD2_DEBUG to one of the numbers listed in src/debug.h (if you want everything, and that IS a lot, just do export BARNYARD2_DEBUG=4294967295).