After playing around with Mr. Roesch’s IP blacklisting patch for Snort, we had noticed that the output wasn’t going to help us as much as we wanted. The problem we were facing was that we use unified output from Snort which is later processed by Barnyard before being placed into the alerts database. When Snort is configured to use unified output, it does so in a binary fashion straight to disk with minimal information to allow for better performace. Then a lower priority process such as Barnyard will read the unified output file and import it to the database. To translate the information in the unified output file, Barnyard relies upon the message map files (gen-msg.map and sid-msg.map) to create more human friendly alerts.
Since the alerts in question are being generated by a preprocessor (src/preprocessors/spp_iplist.c), every alert logged by the preprocessor will just be identified by the entry you create for it in gen-msg.map that is looked up by Barnyard. The preprocessor will only create alerts with a single identifier, so you could update your gen-msg.map file to include something like the following:
136 || 1 || Blacklisted IP
However, if you are using multiple blacklists (such as the ones offered over on Emerging Threats like Russian Business Network aka RBN/bruteforcers/Shadow Server aka botcc/etc) then there is no way for Barnyard to tell which blacklist generated the alert. The unified output will only have the 136 || 1 in it for Barnyard to lookup in the gen-msg.map file.
Mr. Roesch was even so kind as to point out that the solution to this is to update the src/preprocessors/spp_iplist.c and some extra management of the gen-msg.map file. First, update the line in spp_iplist.c from this:
SnortEventqAdd(GENERATOR_SPP_IPLIST, IPLIST_BLACKLIST, 1, 0, 0,
To this:
SnortEventqAdd(GENERATOR_SPP_IPLIST, (int)pn->data, 1, 0, 0,
Instead of just using the value for IPLIST_BLACKLIST from src/generators.h (equal to 1) to signify that the blacklist was true, it will now use the pointer to the integer for the blacklist that was loaded. The numbers are sequential starting from zero and depend upon in what order the blacklist is placed in the snort.conf file. The below is an excerpt of a snort.conf file for use with the ipblacklist preprocessor:
preprocessor iplist: nodrops blacklist bruteforcer /etc/snort/iplists/bruteforceblocker.blacklist \ nodrops blacklist spamhaus /etc/snort/iplists/spamhaus.blacklist \ nodrops blacklist tor-exit /etc/snort/iplists/tor-exitnode.blacklist \ nodrops blacklist tor-server /etc/snort/iplists/tor-server.blacklist \ nodrops blacklist zeus /etc/snort/iplists/zeustracker.blacklist
This would cause the following output during startup (portions omitted for brevity):
Loading bruteforcer blacklist from /etc/snort/iplists/bruteforceblocker.blacklist Loading spamhaus blacklist from /etc/snort/iplists/spamhaus.blacklist Loading tor-exit blacklist from /etc/snort/iplists/tor-exitnode.blacklist Loading tor-server blacklist from /etc/snort/iplists/tor-server.blacklist Loading zeus blacklist from /etc/snort/iplists/zeustracker.blacklist IP List Config: IP Blacklist active with 5241 entries IP Whitelist active with 0 entries Precached event strings: 0 -> Access attempt from bruteforcer blacklisted IP address 1 -> Access attempt from spamhaus blacklisted IP address 2 -> Access attempt from tor-exit blacklisted IP address 3 -> Access attempt from tor-server blacklisted IP address 4 -> Access attempt from zeus blacklisted IP address
So to make sure that the unified output is translated to human readable correctly by Barnyard, we must update the gen-msg.map file appropriately like so:
136 || 0 || iplist: bruteforcer blacklisted ip 136 || 1 || iplist: spamhaus blacklisted ip 136 || 2 || iplist: tor-exit blacklisted ip 136 || 3 || iplist: tor-server blacklisted ip 136 || 4 || iplist: zeus blacklisted ip
This can become somewhat tedious and you would want to ensure that iplists you choose to run don’t change to much (the actual lists, not the contents of them) for consistency sake.
However, after this change we noticed a pretty significant increase in processor load. Upon further investigation we noticed that the IpListEval() function within src/preprocessors/spp_iplist.c was refrencing the same pointer twice when it wasn’t necessary. After some screwing around we were able to drop the CPU utilization by about 10-15% by removing a single pointer refrence. You can see in the below graph the direct increase in the green line after we applied the first change for dealing with the unified output update. You can then see the drop back down after we remove the extra pointer refrence. This small change appears to have such a significant impact as this preprocessor is firing on every UDP/ICMP and TCP (with SYN flag) packet that Snort sees on the wire:
After sharing the information with Mr. Roesch, he updated the IpListEval() function (and so should you if you choose to implement the previous change for updating the unified output):
void IpListEval(Packet *p, void *conext)
{
struct addr saddr;
struct addr daddr;
s_ptrie_node_t *pn = NULL;
int whitelist_detect = 0;
int blacklist_detect = 0;
int bl_ref = 0;
if(!IsIP(p))
{
DEBUG_WRAP(DebugMessage(DEBUG_PLUGIN,
" -> spp_iplist: Not IP\n"););
return;
}
if(((IsTCP(p) && p->tcph->th_flags & TH_SYN)) ||
(IsUDP(p)) || (IsICMP(p)))
{
addr_pack(&saddr, ADDR_TYPE_IP, IP_ADDR_BITS, &p->iph->ip_src,
IP_ADDR_LEN);
addr_pack(&daddr, ADDR_TYPE_IP, IP_ADDR_BITS, &p->iph->ip_dst,
IP_ADDR_LEN);
if(ip_whitelist)
{
if(s_ptrie_find_entry_byaddr(ip_whitelist, &saddr) ||
s_ptrie_find_entry_byaddr(ip_whitelist, &daddr))
{
/* let's bail, should probably set do_detect to 0 too... */
return;
}
}
if(ip_blacklist)
{
if((pn = s_ptrie_find_entry_byaddr(ip_blacklist, &saddr)))
{
blacklist_detect = 1;
bl_ref = (int)pn->data;
}
else if((pn = s_ptrie_find_entry_byaddr(ip_blacklist, &daddr)))
{
blacklist_detect = 1;
bl_ref = (int)pn->data;
}
}
if(blacklist_detect)
{
if(!noalerts)
SnortEventqAdd(GENERATOR_SPP_IPLIST, bl_ref, 1, 0, 0,
list_names[bl_ref], 0);
if(!nodrops && InlineMode())
InlineDrop(p);
}
}
return;
}
This left us with on final issue for our implementation or anyone else who is using this in a passive IDS manner instead of Snort inline. If you recall the lines we had in the snort.conf file pertinent to the iplist preprocessor configuration, we were instructing the preprocessor to not drop the traffic using the nodrops option. The above (and previous) versions of the IpListEval() function are written to fire on any TCP packet that has the SYN flag set. This would include the initial TCP SYN packet from the client and the corresponding TCP SYN/ACK flagged packet from the server. The output would be like the following:
Access attempt from evil blacklisted IP address [**] [Priority: 0] {TCP} 1.1.1.1:80 -> 2.2.2.2:80
Access attempt from evil blacklisted IP address [**] [Priority: 0] {TCP} 2.2.2.2:80 -> 1.1.1.1:80
Now this is duplicating the number of alerts for a single TCP connection. To fix this issue, we updated the IpListEval() function in src/preprocessors/spp_iplist.c from this:
void IpListEval(Packet *p, void *conext)
{
--snip--
if(((IsTCP(p) && p->tcph->th_flags & TH_SYN)) || (IsUDP(p)) || (IsICMP(p)))
--snip--
To this:
void IpListEval(Packet *p, void *conext)
{
--snip--
if(((IsTCP(p) && (TCP_ISFLAGSET(p->tcph, TH_SYN) && !TCP_ISFLAGSET(p->tcph, TH_ACK))) || (IsUDP(p)) || (IsICMP(p))))
--snip--
This change ensures the SYN flag is set, but if it is it will check to ensure the ACK flag is not also set before creating an alert. Now our output looks like this for the same traffic as before:
Access attempt from evil blacklisted IP address [**] [Priority: 0] {TCP} 1.1.1.1:80 -> 2.2.2.2:80
So that is about it, for now. Many thanks to Mr. Roesch for helping me out along they way via the Snort-Devel and Snort-Users mailing lists!