Stephen Reese

This guide walks through configuring SiLK from a source install on a Debian 6 host in order to collect NetFlow data from a Cisco router. The guides here and here written by CERT NetSA are quite good but lack some detail specific to the Debian distribution which required a bit of mucking about to get everything functioning correctly. This assumes that you have a Cisco router to send NetFlow data to a host on your network, in this case, a Debian host.

Installation:

First install a prerequisite.

$ sudo apt-get install libpcap-dev

Next untar and change into the SiLK directory. For Debian I found that using the /usr directory worked well. By default the configure script uses /usr/local in which it places the binaries, libraries, etc outside of Debian’s default paths.

$ ./configure --prefix=/usr --sysconfdir=/etc/silk --enable-data-rootdir=/netflow \
--enable-ipv6 --enable-output-compression

Your output should be something along the following:

    * Configured package:           SiLK 2.4.5
    * Host type:                    x86_64-unknown-linux-gnu
    * Source files ($top_srcdir):   .
    * Install directory:            /usr
    * Root of packed data tree:     /netflow
    * Packing logic:                via run-time plugin
    * Timezone support:             UTC
    * Default compression method:   SK_COMPMETHOD_ZLIB
    * IPv6 support:                 YES
    * IPFIX collection support:     YES (-pthread -lfixbuf -lgthread-2.0 -lrt -lglib-2.0)
    * Transport encryption support: NO (gnutls not found)
    * IPA support:                  NO
    * LIBPCAP support:              YES (-lpcap)
    * ADNS support:                 NO
    * Python support:               NO
    * Build analysis tools:         YES
    * Build packing tools:          YES
    * Compiler (CC):                gcc
    * Compiler flags (CFLAGS):      -I$(srcdir) -I$(top_builddir)/src/include -I$(top_srcdir)/src/include -DNDEBUG -O3 -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wformat=2 -Wdeclaration-after-statement -Wpointer-arith
    * Linker flags (LDFLAGS):
    * Libraries (LIBS):             -lz -ldl -lm

Lastly:

$ make
$ sudo make install

Configuration:

Example files are available in the tarball that you extracted. Modified versions or notes for Debian and similar architectures available below.

/netflow/silk.conf in your data directory, the default is /data but I used /netflow as you can see in the configure toggle above. The changes I made were to reduce the number of sensors.

# The syntactic format of this file
#    version 2 supports sensor descriptions, but otherwise identical to 1
version 2

sensor 0 s0    "Description for sensor S0"
sensor 1 s1

class all
    sensors s0 s1
end class

# Editing above this line is sufficient for sensor definition.

/etc/silk/sensor.conf is the definition for the data coming in from your Cisco router:

probe s0 netflow-v5
    listen-on-port 9990
    protocol udp
    accept-from-host 172.16.0.1
end probe

sensor s0
    netflow-v5-probes s0
    internal-ipblocks 172.16.0.0/24
    external-ipblocks remainder
end sensor

/etc/silk/rwflowpack.conf:

### Packer configuration file  -*- sh -*-
##
## The canonical pathname for this file is /usr/local/etc/rwflowpack.conf
##
## RCSIDENT("$SiLK: rwflowpack.conf.in 16306 2010-09-15 18:14:41Z mthomas $")
##
## This is a /bin/sh file that gets loaded by the init.d/rwflowpack
## wrapper script, and this file must follow /bin/sh syntax rules.

# Set to non-empty value to enable rwflowpack
ENABLED=yes

# These are convenience variables for setting other values in this
# configuration file; their use is not required.
statedirectory=/var/lib/rwflowpack

# If CREATE_DIRECTORIES is set to "yes", the directories named in this
# file will be created automatically if they do not already exist
CREATE_DIRECTORIES=yes

# Full path of the directory containing the "rwflowpack" program
BIN_DIR=/usr/sbin

# The full path to the sensor configuration file.  Used by
# --sensor-configuration.  YOU MUST PROVIDE THIS (the value is ignored
# when INPUT_MODE is "respool").
SENSOR_CONFIG=/etc/silk/sensor.conf

# The full path to the root of the tree under which the packed SiLK
# Flow files will be written.  Used by --root-directory.
DATA_ROOTDIR=/netflow

# The full path to the site configuration file.  Used by
# --site-config-file.  If not set, defaults to silk.conf in the
# ${DATA_ROOTDIR}.
SITE_CONFIG=/netflow/silk.conf

# Specify the path to the packing-logic plug-in that rwflowpack should
# load and use.  The plug-in provides functions that determine into
# which class and type each flow record will be categorized and the
# format of the files that rwflowpack will write.  When SiLK has been
# configured with hard-coded packing logic (i.e., when
# --enable-packing-logic was specified to the configure script), this
# value should be empty.  A default value for this switch may be
# specified in the ${SITE_CONFIG} site configuration file.  This value
# is ignored when INPUT_MODE is "respool".
PACKING_LOGIC=

# Data input mode.  Valid values are:
#  * "stream" mode to read from the network or from probes that have
#    poll-directories
#  * "fcfiles" to process flowcap files on the local disk
#  * "respool" to process SiLK flow files maintaining the sensor and
#    class/type values that already exist on those records.
INPUT_MODE=stream

# Directory in which to look for incoming flowcap files in "fcfiles"
# mode or for incoming SiLK files in "respool" mode
INCOMING_DIR=${statedirectory}/incoming

# Directory to move input files to after successful processing.  When
# in "stream" mode, these are the files passed to any probe with a
# poll-directory directive.  When in "fcfiles" mode, these are the
# flowcap files.  When in "respool" mode, these are the SiLK Flow
# files.  If not set, the input files are not archived but are deleted
# instead.
ARCHIVE_DIR=${statedirectory}/archive

# When using the ARCHIVE_DIR, normally files are stored in
# subdirectories of the ARCHIVE_DIR.  If this variable's value is 1,
# files are stored in ARCHIVE_DIR itself, not in subdirectories of it.
FLAT_ARCHIVE=0

# Directory to move an input file into if there is a problem opening
# the file.  If this value is not set, rwflowpack will exit when it
# encounters a problem file.  When in "fcfiles" mode, these are the
# flowcap files.  When in "stream" mode, these are the files passed to
# any probe with a poll-directory directive.
ERROR_DIR=  #${statedirectory}/error

# Data output mode.  Valid values are "local" and "remote".  "local"
# writes the hourly data files to the local disk.  "remote" creates
# small files (called incremental files) that must be processed by
# rwflowappend to create the hourly files.
OUTPUT_MODE=local

# Directory in which the incremental files are written when the
# OUTPUT_MODE is "remote".  Typically there is an rwsender deamon that
# polls this directory for new incremental files.
SENDER_DIR=${statedirectory}/sender-incoming

# Temporary directory in which to build incremental files prior to
# handing them to rwsender.  Used only when OUTPUT_MODE is "remote".
INCREMENTAL_DIR=${statedirectory}/incremental

# The type of compression to use for packed files.  Left empty, the
# value chosen at compilation time will be used.  Valid values are
# "best" and "none".  Other values are system-specific (the available
# values are listed in the description of the --compression-method
# switch in the output of rwflowpack --help).
COMPRESSION_TYPE=best

# Interval between attempts to check the INCOMING_DIR or
# poll-directory probe entries for new files, in seconds.  This may be
# left blank, and will default to 15.
POLLING_INTERVAL=

# Interval between periodic flushes of open SiLK Flow files to disk,
# in seconds.  This may be left blank, and will default to 120.
FLUSH_TIMEOUT=

# Maximum number of SiLK Flow files to have open for writing
# simultaneously.  This may be left blank, and will default to 64
FILE_CACHE_SIZE=

# Whether rwflowpack should use advisory write locks.  1=yes, 0=no.
# Set to zero if messages like "Cannot get a write lock on file"
# appear in rwflowpack's log file.
FILE_LOCKING=1

# Whether rwflowpack should include the input and output SNMP
# interfaces and the next-hop-ip in the output files.  1=yes, 0=no.
# The default is no, and these values are not stored to save disk
# space.  (The input and output fields contain VLAN tags when the
# sensor.conf file contains the attribute "interface-values vlan".)
PACK_INTERFACES=0

###

# The type of logging to use.  Valid values are "legacy" and "syslog".
LOG_TYPE=syslog

# The lowest level of logging to actually log.  Valid values are:
# emerg, alert, crit, err, warning, notice, info, debug
LOG_LEVEL=info

# The full path of the directory where the log files will be written
# when LOG_TYPE is "legacy".
LOG_DIR=/var/log

# The full path of the directory where the PID file will be written
PID_DIR=${LOG_DIR}

# The user this program runs as; root permission is required only when
# rwflowpack listens on a privileged port.
USER=root
#USER=`whoami`  # run as user invoking the script

# Extra options to pass to rwflowpack
EXTRA_OPTIONS=

/etc/init.d/rwflowback directory, the only change was to line 38 in order to change to the configuration specified in the configure statement.

SCRIPT_CONFIG_LOCATION="/etc/silk" 

With everything installed in their respective locations it is time to move on to setting up the Cisco device.

Router(config)# ip cef
Router(config)# ip flow-export source Loopback0
Router(config)# ip flow-export version 5
Router(config)# ip flow-export destination x.x.x.x 9990
Router(config)# interface  f1/0
Router(config-if)# ip flow ingress
Router(config-if)# ip flow egress 

I hope this helps. If you have any comments or questions, leave a comment below.