CESNET Intrusion Detection System

CESNET technical report number 10/2007
also available in PDF, PostScript, and XML formats.

Pavel Vachek
December 6, 2007

Keywords: IDS, Instrusion Detection System, LaBrea, CESNET, SuSE Linux

1   Abstract

This technical report describes a simple system for detecting intrusion attempts caused by hackers and viruses. Its main component is the LaBrea program distributed under GPL licence, running on a PC server under SuSE Linux. This author's programs allow automated distribution of warning e-mails to administrators of networks where the attacks originated.

2   Introduction

A simple and useful Intrusion Detection System (IDS) has been developed in CESNET, the National Research and Educational Network of the Czech Republic. It has been operating since December 2002, informing all administrators from selected networks (hereinafter, from own networks) about any TCP attacks originating there. As a result, those administrators are informed soon; they can check the potentially compromised machines, disinfect them or disconnect them from the Internet, and thus keep up the CESNET reputation. - Attacks originating outside these own networks are reported to DSHIELD, a global cooperative security system.

The goal of this Technical Report is to inform other network managers, especially those of University networks with large IP address allocations, about the system usefulness and to help them install a similar system to make their everyday duties much easier.

3   The IDS Project

Main components of the CESNET IDS are:

IDS helper tools are:

All helper tools are also described in detail below. The CESNET IDS data flow is shown in the following diagram.

[Figure]

Figure 1: CESNET IDS Data Flow Diagram

4   The LaBrea Program

Goal of the LaBrea program is to stop or at least slow down propagation of network viruses and hacker activities. Its author wrote it in response to the Code Red virus which started to propagate in July 2001. Installing just the LaBrea server in standalone mode helps the Internet enough; however, it is much more useful after the complete IDS described here has been set up.

Main features of the LaBrea program are:

LaBrea responds to TCP connection attempts as follows:

5   Installation

Suppose that our Institution has been allocated an address range 172.16.0.0/16, i.e., 65536 IP addresses. Also suppose that subnet 172.16.224.0/22, i.e., 1024 IP addresses, has not yet been assigned and will never be. This subnet will be reserved for our IDS interface Eth1.

The IDS interface Eth0 is assigned an address from another part of our network, e.g., 172.16.1.2. This is the only IP address advertised in DNS which needs a reverse (PTR) record. It is used for sending e-mail to administrators of own networks whose machines attempted to connect to the subnet monitored by the Eth1, and to send data on attacks from worldwide Internet to the DSHIELD project. The Eth0 interface can also be accessed using SSH from some selected addresses.

Before the LaBrea installation starts, one should check if the Linux distribution used includes programs Gcc, Perl, Libpcap, and Whois. The latter program is a part of SuSE Linux and several other distributions. It can be downloaded from Debian repository. Current version is 4.7.23 (60 kB; Sep 12, 2007).

Latest version of program Libdnet is 1.11 (446 kB; Jan 19, 2006) and latest version of LaBrea is 2.5-stable-1 (210 kB; Oct 29, 2003).

Using a well-known command sequence

$ ./configure
$ make
$ su
$ make install

the Libdnet and LaBrea programs should be installed.

First, LaBrea should be run manually in Test mode (only Eth0 interface will be used, network traffic will not be influenced):

$ labrea -T -z

LaBrea should report something like this:

Fri Oct 5 16:01:31 2007  LaBrea will attempt to capture unused IPs.
Fri Oct 5 16:01:31 2007  Full internal BPF filter: arp or
  (ip and ether dst host 00:00:0F:FF:FF:FF)
Fri Oct 5 16:01:31 2007  LaBrea will log to stdout
Fri Oct 5 16:01:31 2007  Logging will be very verbose.
Fri Oct 5 16:01:31 2007  Initiated on interface: eth0
Fri Oct 5 16:01:31 2007  Host system IP addr: 172.16.1.2,
  MAC addr: 00:12:34:56:78:9a
Fri Oct 5 16:01:31 2007  ... Processing configuration file
Fri Oct 5 16:01:31 2007  ... End of configuration file
  processing
Fri Oct 5 16:01:31 2007  Network number: 172.16.1.0
Fri Oct 5 16:01:31 2007  Netmask: 255.255.255.128
Fri Oct 5 16:01:31 2007  Number of addresses LaBrea will watch for ARPs: 127
Fri Oct 5 16:01:31 2007  Range: 172.16.1.0 - 172.16.1.127
Fri Oct 5 16:01:31 2007  Throttle size set to WIN 10
Fri Oct 5 16:01:31 2007  Rate (-r) set to 3
Fri Oct 5 16:01:31 2007  Test mode run complete...
  LaBrea is exiting.
Fri Oct 5 16:01:31 2007  LaBrea exiting...
Fri Oct 5 16:01:31 2007  35/0 packets (received/dropped) by filter

In the next step, IDS should use both Ethernet interfaces:

Again, LaBrea is started manually:

$ labrea -b -H -i eth1 -I 172.16.225.77 -no-arp-sweep \
  -o -p 2400 -v -z

LaBrea should be responding to TCP connection attempts, ICMP Echo (Ping) and SYN+ACKs received. If everything works, Eth1 can be set up as unnumbered interface which further improves the IDS server security. The following paragraph which describes this procedure will help especially users of the SuSE Linux distribution; users of other distributions must use similar functions available in their operating systems.

5.1   Setting up unnumbered Eth1 interface

Original status of unconfigured Eth1 interface may be something like:

$ ifconfig eth1
eth1  Link encap:Ethernet  HWaddr 00:09:87:65:43:21
      BROADCAST MULTICAST  MTU:1500  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000
      RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

Using the YaST tool, appropriate interface parameters should be set up. The IP address can be any from the monitored range, e.g., 172.16.225.77 again. Afterwards, ifconfig will report something like:

$ ifconfig eth1
eth1  Link encap:Ethernet  HWaddr 00:09:87:65:43:21
      inet addr:172.16.225.77  Bcast:172.16.227.255  Mask:255.255.252.0
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:11 errors:0 dropped:0 overruns:0 frame:0
      TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000
      RX bytes:660 (660.0 b)  TX bytes:460 (460.0 b)

Configuration file for this interface located in the /etc/sysconfig/network directory will contain the following:

$ more ifcfg-eth-id-00\:09:87\:65\:43\:21
BOOTPROTO='static'
BROADCAST='172.16.227.255'
IPADDR='172.16.225.77'
MTU=''
NETMASK='255.255.252.0'
NETWORK='172.16.224.0'
REMOTE_IPADDR=''
STARTMODE='onboot'
UNIQUE='Hkyc.MvtRJ6g27Q1'
_nm_name='bus-pci-0000:01:0a.0'

Attention: To make a backup copy of this file, one must change the start of its filename - e.g., to sav-ifcfg-eth-id-00\:09\:87\:65\:43\:21.

The BROADCAST, IPADDR, NETMASK a NETWORK data should be cleared using a simple text editor. Afterwards, this file will contain the following:

$ more ifcfg-eth-id-00\:09\:87\:65\:43\:21
BOOTPROTO='static'
BROADCAST=''
IPADDR=''
MTU=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='onboot'
UNIQUE='Hkyc.MvtRJ6g27Q1'
_nm_name='bus-pci-0000:01:0a.0'

Operating system or the network subsystem must be restarted. The following data should appear in the log:

Setting up network interfaces:
    lo
    lo        IP address: 127.0.0.1/8   .  .  .  .  .  .  .  .  .  .  done
Waiting for mandatory devices:
  eth-id-00:12:34:56:78:9a eth-id-00:09:87:65:43:21
    eth0      device: AAAAA Corp. [Ethernet ABCD] (rev 08)
    eth0      configuration: eth-id-00:12:34:56:78:9a
    eth0      IP address: 172.16.1.2/25
    eth1      device: BBBBB Corp. [Ethernet EFGH] (rev 09)
    eth1      configuration: eth-id-00:09:87:65:43:21
Setting up service network  .  .  .  .  .  .  .  .  .  done

One can see that the Eth1 interface is functional but no IP address is assigned. Program ifconfig will display its status as follows:

$ ifconfig eth1
eth1  Link encap:Ethernet  HWaddr 00:09:87:65:43:21
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:442 errors:0 dropped:0 overruns:0 frame:0
      TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000
      RX bytes:3199 (3.1 Kb)  TX bytes:1005 (1.0 Kb)

Program LaBrea can now be launched using the following command line (for detailed information on command-line options please see the man labrea output):

$ labrea -b -E 00:09:87:65:43:21 -H -i eth1 \
  -I 172.16.225.77 -n 172.16.224.0/22 -no-arp-sweep \
  -O -o -p 2400 -v -z

Program LaBrea should run now for some time to allow administrators to become acquainted with its features (command-line and/or config file parameters). One can try to connect to the monitored range using, e.g., the Telnet program, test how LaBrea responds to the Ping command, learn about the bandwidth consumed, frequency of attacks, number of DDoS attacks seeming to originate from the monitored subnet, etc.

5.2   Installing the remaining IDS components

One will find out that LaBrea generates too much unnecessary data, and most of it is coming out of foreign networks which are not too important for us. Therefore, the LaBrea output data will be processed in two ways: data on attacks from our own networks will be processed directly by our IDS while the remaining data will be passed over to DSHIELD for further processing.

The data to be processed directly should contain only the following:

Appropriate configuration data can be found in file LBstrings.

The data to be passed over to DSHIELD will contain similar information but data on attacks includes all connection attempts from the worldwide Internet. Appropriate configuration data can be found in file LBstrings.all.

One must also make sure that

This is the purpose of file /etc/init.d/labrea which should be installed using the command 

$ insserv -v labrea

CESNET IDS processes the LaBrea output data at times specified by the crontab file of user root. The following operations are performed:

Programs LaBreaBackEnd and LaBreaReport are written in Perl. Both of them require module Mail::Sender to be installed. Currently, its latest version is 0.8.13, dated Feb 25, 2006. It can be installed using the following commands:

$ perl Makefile.PL
$ make
$ make test
$ make install

Several default values must be set up during Mail::Sender installation. The most important one is the mail server address. If Postfix runs on the IDS server, its address can be given as localhost. All other questions can be answered by pressing Enter or (in case of default coding) by pressing n.

The LaBreaBackEnd in the CESNET IDS is launched as follows:

$ LaBreaBackEnd -b -i file1 -l limit -O -v

-b    batch mode
-i file1   file1 = input filename (LaBrea output log)
-l limit   attacks below `limit' from a single IP address
             are ignored
           limit = 0 ... all attacks are processed
-O         only attacks from networks listed in @ownnet
             are processed
-v         verbose output.

One purpose of the LaBreaBackEnd is to search for e-mail addresses of all administrators responsible for networks where the attacks have originated. LaBreaBackEnd also finds all administrators responsible for the originating domain if a reverse record for the IP address in question is found. Program selects the most appropriate address(es) from the resulting list. Preferred addresses are generated by LaBreaBackEnd and added to the list if none is found in the list.

LaBreaBackEnd minimises the number of warning e-mails sent: Should LaBrea detect attacks from different networks managed by a single set of administrators, a single e-mail with all appropriate error reports is sent to this set.

Log file yymmdd.log is generated by LaBreaBackEnd with detailed data on program run and results. Should LaBrea detect any attacks from own networks, the following is also created:

LaBreaBackEnd also sends the IDS system administrator an SMS text message containing the number of format errors in the input data, number of attacking machines and number of generated data files.

The LaBreaReport program is launched only if LaBreaBackEnd has generated any data files. LaBreaReport reads the files in the result directory and sends them by e-mail after formatting them properly and after adding some explanatory notes. Implicitly, these files are deleted afterwards. Various command-line options can be used; the following are used by the CESNET IDS:

$ LaBreaReport -k -i
-k	sent files are kept (not deleted), renamed to `*.sent'
-i	files `*.sent' in `result' directory are ignored.

The following command-line parameters can additionally be used for testing LaBreaReport:

-t	no e-mails are sent, no files are deleted
-T	e-mails are sent only to $testaddr
-v	verbose mode.

LaBreaBackEnd contains some variables which should be checked and/or set up. These are especially the following:

@ignoredom    no warning e-mail is sent to administrators
                of domains listed in @ignoredom
@ndom         attacks from domains listed in @ndom
                are ignored
@nipadd       attacks from single IP addresses listed in
                @nipadd are ignored
@nipnet       attacks from networks listed in @nipnet
                are ignored
@ownnet       only own networks listed in @ownnet
                will be monitored if command-line option -O
                is selected
$smsadd       e-mail address where the summary SMS message
                is to be sent.

Exact IP addresses of the monitored network should not be displayed in the warning messages. The following two variables will help:

$strin        string (part of destination IP address)
                to be replaced ...
$strout       ... by this string.

The following LaBreaReport variables should also be set up:

$bccaddr      blind copies (Bcc:) are sent here unless the
                -t or -T command-line options are selected
$testaddr     address where warning e-mails are sent if -T
                option is selected.

6   Recommended IDS configuration

The following long-time experience may help in choosing a proper IDS server hardware:

Currently, the CESNET IDS runs on a DELL PowerEdge 1425SC in the following configuration:

The IDS should be located as close to the Institution border router as possible. Complete TCP traffic should enter the IDS - no TCP ports should be filtered. No switch should be used to connect the IDS if large subnet is to be monitored (its ARP table might overflow).

7   Experience Gained

Originally, the CESNET IDS was designed to detect and report attacks from the worldwide Internet. As a result, many of its features are not currently used but its author wishes to keep them for those who may find them useful.

The CESNET IDS has concentrated on attacks from its own networks for administrative, not for technical reasons. As soon as the IDS started sending out its warning worldwide in 2002, their recipients (several thousand per week) started to respond as follows:

  1. Appreciated the warning - usually autoresponders
  2. Appreciated the warning but sent a list of domains or IP addresses to be ignored.
  3. Did not appreciate the reports but sent a correct destination e-mail address.
  4. Did not appreciate the reports - regarded them as spam.
  5. Responded with virus-infected mail and/or spam.

As a result, the CESNET IDS was redesigned to cope with requirements 2, 3 and 4. Still, monitoring the worldwide Internet attacks brings a big problem: The IDS administrator would have to spend several hours each day communicating with network administrators and updating the LaBreaBackEnd configuration according to their wishes. Another big problem are the missing or obsolete administrator addresses as well as abuse addresses according to RFC 2142 (abuse@example.net). Much of the IDS administrator's work is futile if the network/domain administrator contact addresses are not updated regularly in the ARIN, APNIC, RIPE DB etc.

This is why the author of CESNET IDS decided to concentrate on distributing the warning e-mails only to the CESNET network where he can persuade the network managers to keep their contact data in the RIPE DB up-to-date and to set up the abuse addresses, too. The remaining data on attacks from the worldwide Internet is sent by e-mail to the DSHIELD project which handles it appropriately.

7.1   Advantages

7.2   Disadvantages

8   Conclusion

The CESNET IDS described above is used for monitoring a single /16 address allocation. This is also the basic configuration suitable for most large institutions. During 2007, this Intrusion Detection System recorded 204145 connection attempts coming from 222 compromised machines within the CESNET2 networks. Thanks to the IDS design, only 190 warning messages have been sent in 167 batches to their managers. - In addition, countless other connection attempts from the rest of the Internet have been reported to the DSHIELD.

CESNET is a NREN with several independent large allocations, and all of them should be monitored. Therefore, for several months the CESNET IDS has been running in an advanced mode where two different large allocations are being monitored by a single IDS server. To achieve this, two independent instances of LaBrea are running and only simple changes in configuration files are necessary. Interestingly, the two Ethernet interfaces described above are sufficient for this task.

This author will be glad to learn that a similar new IDS has been installed.

9   Appendix: Configuration files

9.1   File crontab for user root

$ crontab -l
# Every day at 0.04, 2.04, 4.04, ..., 22.04 GMT:
4 */2 * * * /root/bin/DShield.sh
# Every Mon - Fri at 6.05,8.05,10.05,12.05,14.05,16.05 GMT:
5 6-16/2 * * 1-5 /root/bin/LBbe.sh

9.2   File DShield.sh

#! /bin/sh
LABREA_LOG_ALL=/var/log/LaBrea.ALL
NICE_BIN=/usr/bin/nice
DSHIELD_LABREA=/root/bin/labrea.pl
DATESTRING=`/bin/date +%s`
LABREA_START=/var/log/START.$DATESTRING
TOUCH_BIN=/usr/bin/touch

/etc/init.d/labrea restart
# echo "Creating the START.$DATESTRING timestamp..."
$TOUCH_BIN $LABREA_START
echo -n "Processing, sending and deleting DSHIELD log ("
$NICE_BIN -n 100 $DSHIELD_LABREA
LABREA_LEN=`ls -lh $LABREA_LOG_ALL |cut -d" " -f6`
echo -n "$LABREA_LEN bytes)....."
rm -f $LABREA_LOG_ALL
echo "Done."

9.3   File LBbe.sh

#! /bin/sh
BZIP2_BIN=/usr/bin/bzip2
DATESTRING=`/bin/date +%y%m%d%H`
LBBE_LNK=/root/bin/LBbe
LBREP_LNK=/root/bin/LBrep
LABREA_LOG=/var/log/LaBrea
LABREA_LOG_SAV=/var/log/LaBrea.$DATESTRING

echo "Processing the LaBrea log file, sending results (if any)..."
$LBBE_LNK -b -l 0 -i $LABREA_LOG -O -v && $LBREP_LNK -k -i
echo -n "Renaming "
mv -f -backup=t $LABREA_LOG $LABREA_LOG_SAV
echo -n "and compressing the LaBrea log file... "
$BZIP2_BIN $LABREA_LOG_SAV
echo "Done."

9.4   File LBstrings

ting: 172\.16\.
(...)
average bw:
SYN/ACK:
to a Ping

9.5   File LBstrings.all

tarpitting:
average bw:
SYN/ACK:
to a Ping

9.6   File /etc/init.d/labrea

#! /bin/sh
# /etc/init.d/labrea
### BEGIN INIT INFO
# Provides: labrea
# Required-Start: $network
# Required-Stop: $network
# Default-Start: 3 5
# Default-Stop: 3 5
# Description: Start the LaBrea program
### END INIT INFO
LABREA_BIN=/usr/local/sbin/labrea
LABREA_LOG=/var/log/LaBrea
LABREA_LOG_ALL=/var/log/LaBrea.ALL
test -x $LABREA_BIN || exit 5
GREP_BIN=/bin/grep
test -x $GREP_BIN || exit 5
TEE_BIN=/usr/bin/tee
test -x $TEE_BIN || exit 5

. /etc/rc.status
rc_reset
case "$1" in
    start)
        echo -n "Starting LaBrea 2.51, appending to existing log"
        /sbin/startproc -f $LABREA_BIN -b -E 00:09:87:65:43:21 -H -i eth1 \
          -I 172.16.225.77 -n 172.16.224.0/22 -no-arp-sweep -O -o \
          -p 2400 -R -v -z| $GREP_BIN -f /var/log/LBstrings.all| \
          $TEE_BIN -a /var/log/LaBrea.all|$GREP_BIN \
          -f /var/log/LBstrings >> /var/log/LaBrea.new &
        rc_status -v
        ;;
    stop)
        echo -n "Shutting down LaBrea 2.51..."
        /sbin/killproc -TERM $LABREA_BIN
        rc_status -v
        ;;
    restart)
        echo -n "Shutting down LaBrea 2.51..."
        /sbin/killproc -TERM $LABREA_BIN
        rc_status -v
        echo "Renaming the resulting log files..."
        mv -f /var/log/LaBrea.new $LABREA_LOG
        mv -f /var/log/LaBrea.all $LABREA_LOG_ALL
        echo -n "Starting LaBrea 2.51, opening new log ..."
        /sbin/startproc -f $LABREA_BIN -b \
          -E 00:09:87:65:43:21 -H -i eth1 -I 172.16.225.77 \
          -n 172.16.224.0/22 -no-arp-sweep -O -o -p 2400 \
          -R -v -z |$GREP_BIN -f /var/log/LBstrings.all| \
          $TEE_BIN /var/log/LaBrea.all|$GREP_BIN -f \
          /var/log/LBstrings > /var/log/LaBrea.new &
        rc_status -v
        ;;
    status)
        LCNTNEW=`wc -l /var/log/LaBrea.new | cut -f1 -d" "`
        LCNTALL=`wc -l /var/log/LaBrea.all | cut -f1 -d" "`
        echo -n "LaBrea 2.51: $LCNTNEW LaBrea.new, $LCNTALL LaBrea.all lines"
        /sbin/checkproc $LABREA_BIN
        rc_status -v
        ;;
    *)
        echo "Usage: $0 {start|stop|status|restart}"
        exit 1
        ;;
esac
rc_exit

9.7   File /etc/dshield.cnf

from=...
userid=...
to=reports@dshield.org
cc=
bcc=
log=/var/log/LaBrea.ALL
line_filter=Initial Connect
sendmail=/usr/sbin/sendmail -oi -t
whereto=MAIL
source_exclude=/etc/dshield-source-exclude.lst
target_exclude=/etc/dshield-target-exclude.lst
source_port_exclude=/etc/dshield-source-port-exclude.lst
target_port_exclude=/etc/dshield-target-port-exclude.lst
obfus=N
linecnt=/tmp/dshield.cnt
verbose=N
debug=N
rotate=N
tz=+00:00

9.8   File /etc/dshield-source-exclude.lst

172.16.0.0/16

9.9   Other files

Files /etc/dshield-target-exclude.lst, /etc/dshield-source-port-exclude.lst, /etc/dshield-target-port-exclude.lst and /usr/local/etc/labrea.conf are empty.

Files LaBreaBackEnd, LaBreaReport and labrea.pl are too long to be published here. They can be found at address ftp://ftp.cesnet.cz/local/ids signed with PGP key 1BEDF25D1.

References

[Dsh07] How to submit your firewall logs to DShield. Available online.
[Vac03] Vachek P.: Zabezpečení lokálních sítí CESNET2. In: Vysokorychlostní síť národního výzkumu a její nové aplikace, p. 228-231. Praha: CESNET, 2003. Available online.
[Vac06] Vachek P.: CESNET Intrusion Detection System. Technical report 5/2006, Praha: CESNET, 2006.

 

Footnotes:

  1. Pavel.Vachek@cesnet.cz, fingerprint 6627 E8B9 29EC A28A EBE3 8A7F 44C4 BABC 1BED F25D

další weby:fond rozvojemetacentrumCzechLightpřenosyvideoservereduroameduID.cz