Configure SiLK on Linux for NetFlow collection from a Cisco router

This guide walks through configuring SiLK from a source install on a Debian 6 host in order to collect NetFlow data from a Cisco router. The guides here and here written by CERT NetSA are quite good but lack some detail specific to the Debian distribution which required a bit of mucking about to get everything functioning correctly. This assumes that you have a Cisco router to send NetFlow data to a host on your network, in this case, a Debian host.

Installation:

First install a prerequisite.

$ sudo apt-get install libpcap-dev

Next untar and change into the SiLK directory. For Debian I found that using the /usr directory worked well. By default the configure script uses /usr/local in which it places the binaries, libraries, etc outside of Debian’s default paths.

$ ./configure --prefix=/usr --sysconfdir=/etc/silk --enable-data-rootdir=/netflow \
--enable-ipv6 --enable-output-compression

Your output should be something along the following:

    * Configured package:           SiLK 2.4.5
    * Host type:                    x86_64-unknown-linux-gnu
    * Source files ($top_srcdir):   .
    * Install directory:            /usr
    * Root of packed data tree:     /netflow
    * Packing logic:                via run-time plugin
    * Timezone support:             UTC
    * Default compression method:   SK_COMPMETHOD_ZLIB
    * IPv6 support:                 YES
    * IPFIX collection support:     YES (-pthread -lfixbuf -lgthread-2.0 -lrt -lglib-2.0)
    * Transport encryption support: NO (gnutls not found)
    * IPA support:                  NO
    * LIBPCAP support:              YES (-lpcap)
    * ADNS support:                 NO
    * Python support:               NO
    * Build analysis tools:         YES
    * Build packing tools:          YES
    * Compiler (CC):                gcc
    * Compiler flags (CFLAGS):      -I$(srcdir) -I$(top_builddir)/src/include -I$(top_srcdir)/src/include -DNDEBUG -O3 -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wformat=2 -Wdeclaration-after-statement -Wpointer-arith
    * Linker flags (LDFLAGS):
    * Libraries (LIBS):             -lz -ldl -lm

Lastly:

$ make
$ sudo make install

Configuration:

Example files are available in the tarball that you extracted. Modified versions or notes for Debian and similar architectures available below.

/netflow/silk.conf in your data directory, the default is /data but I used /netflow as you can see in the configure toggle above. The changes I made were to reduce the number of sensors.


# The syntactic format of this file
#    version 2 supports sensor descriptions, but otherwise identical to 1
version 2

sensor 0 s0    "Description for sensor S0"
sensor 1 s1

class all
    sensors s0 s1
end class

# Editing above this line is sufficient for sensor definition.

/etc/silk/sensor.conf is the definition for the data coming in from your Cisco router:


probe s0 netflow-v5
    listen-on-port 9990
    protocol udp
    accept-from-host 172.16.0.1
end probe

sensor s0
    netflow-v5-probes s0
    internal-ipblocks 172.16.0.0/24
    external-ipblocks remainder
end sensor

/etc/silk/rwflowpack.conf:


### Packer configuration file  -*- sh -*-
##
## The canonical pathname for this file is /usr/local/etc/rwflowpack.conf
##
## RCSIDENT("$SiLK: rwflowpack.conf.in 16306 2010-09-15 18:14:41Z mthomas $")
##
## This is a /bin/sh file that gets loaded by the init.d/rwflowpack
## wrapper script, and this file must follow /bin/sh syntax rules.

# Set to non-empty value to enable rwflowpack
ENABLED=yes

# These are convenience variables for setting other values in this
# configuration file; their use is not required.
statedirectory=/var/lib/rwflowpack

# If CREATE_DIRECTORIES is set to "yes", the directories named in this
# file will be created automatically if they do not already exist
CREATE_DIRECTORIES=yes

# Full path of the directory containing the "rwflowpack" program
BIN_DIR=/usr/sbin

# The full path to the sensor configuration file.  Used by
# --sensor-configuration.  YOU MUST PROVIDE THIS (the value is ignored
# when INPUT_MODE is "respool").
SENSOR_CONFIG=/etc/silk/sensor.conf

# The full path to the root of the tree under which the packed SiLK
# Flow files will be written.  Used by --root-directory.
DATA_ROOTDIR=/netflow

# The full path to the site configuration file.  Used by
# --site-config-file.  If not set, defaults to silk.conf in the
# ${DATA_ROOTDIR}.
SITE_CONFIG=/netflow/silk.conf

# Specify the path to the packing-logic plug-in that rwflowpack should
# load and use.  The plug-in provides functions that determine into
# which class and type each flow record will be categorized and the
# format of the files that rwflowpack will write.  When SiLK has been
# configured with hard-coded packing logic (i.e., when
# --enable-packing-logic was specified to the configure script), this
# value should be empty.  A default value for this switch may be
# specified in the ${SITE_CONFIG} site configuration file.  This value
# is ignored when INPUT_MODE is "respool".
PACKING_LOGIC=

# Data input mode.  Valid values are:
#  * "stream" mode to read from the network or from probes that have
#    poll-directories
#  * "fcfiles" to process flowcap files on the local disk
#  * "respool" to process SiLK flow files maintaining the sensor and
#    class/type values that already exist on those records.
INPUT_MODE=stream

# Directory in which to look for incoming flowcap files in "fcfiles"
# mode or for incoming SiLK files in "respool" mode
INCOMING_DIR=${statedirectory}/incoming

# Directory to move input files to after successful processing.  When
# in "stream" mode, these are the files passed to any probe with a
# poll-directory directive.  When in "fcfiles" mode, these are the
# flowcap files.  When in "respool" mode, these are the SiLK Flow
# files.  If not set, the input files are not archived but are deleted
# instead.
ARCHIVE_DIR=${statedirectory}/archive

# When using the ARCHIVE_DIR, normally files are stored in
# subdirectories of the ARCHIVE_DIR.  If this variable's value is 1,
# files are stored in ARCHIVE_DIR itself, not in subdirectories of it.
FLAT_ARCHIVE=0

# Directory to move an input file into if there is a problem opening
# the file.  If this value is not set, rwflowpack will exit when it
# encounters a problem file.  When in "fcfiles" mode, these are the
# flowcap files.  When in "stream" mode, these are the files passed to
# any probe with a poll-directory directive.
ERROR_DIR=  #${statedirectory}/error

# Data output mode.  Valid values are "local" and "remote".  "local"
# writes the hourly data files to the local disk.  "remote" creates
# small files (called incremental files) that must be processed by
# rwflowappend to create the hourly files.
OUTPUT_MODE=local

# Directory in which the incremental files are written when the
# OUTPUT_MODE is "remote".  Typically there is an rwsender deamon that
# polls this directory for new incremental files.
SENDER_DIR=${statedirectory}/sender-incoming

# Temporary directory in which to build incremental files prior to
# handing them to rwsender.  Used only when OUTPUT_MODE is "remote".
INCREMENTAL_DIR=${statedirectory}/incremental


# The type of compression to use for packed files.  Left empty, the
# value chosen at compilation time will be used.  Valid values are
# "best" and "none".  Other values are system-specific (the available
# values are listed in the description of the --compression-method
# switch in the output of rwflowpack --help).
COMPRESSION_TYPE=best

# Interval between attempts to check the INCOMING_DIR or
# poll-directory probe entries for new files, in seconds.  This may be
# left blank, and will default to 15.
POLLING_INTERVAL=

# Interval between periodic flushes of open SiLK Flow files to disk,
# in seconds.  This may be left blank, and will default to 120.
FLUSH_TIMEOUT=

# Maximum number of SiLK Flow files to have open for writing
# simultaneously.  This may be left blank, and will default to 64
FILE_CACHE_SIZE=

# Whether rwflowpack should use advisory write locks.  1=yes, 0=no.
# Set to zero if messages like "Cannot get a write lock on file"
# appear in rwflowpack's log file.
FILE_LOCKING=1

# Whether rwflowpack should include the input and output SNMP
# interfaces and the next-hop-ip in the output files.  1=yes, 0=no.
# The default is no, and these values are not stored to save disk
# space.  (The input and output fields contain VLAN tags when the
# sensor.conf file contains the attribute "interface-values vlan".)
PACK_INTERFACES=0


###

# The type of logging to use.  Valid values are "legacy" and "syslog".
LOG_TYPE=syslog

# The lowest level of logging to actually log.  Valid values are:
# emerg, alert, crit, err, warning, notice, info, debug
LOG_LEVEL=info

# The full path of the directory where the log files will be written
# when LOG_TYPE is "legacy".
LOG_DIR=/var/log

# The full path of the directory where the PID file will be written
PID_DIR=${LOG_DIR}

# The user this program runs as; root permission is required only when
# rwflowpack listens on a privileged port.
USER=root
#USER=`whoami`  # run as user invoking the script

# Extra options to pass to rwflowpack
EXTRA_OPTIONS=

/etc/init.d/rwflowback directory, the only change was to line 38 in order to change to the configuration specified in the configure statement.

SCRIPT_CONFIG_LOCATION="/etc/silk" 

With everything installed in their respective locations it is time to move on to setting up the Cisco device.

Router(config)# ip cef 
Router(config)# ip flow-export source Loopback0 
Router(config)# ip flow-export version 5 
Router(config)# ip flow-export destination x.x.x.x 9990 
Router(config)# interface  f1/0 
Router(config-if)# ip flow ingress 
Router(config-if)# ip flow egress 

I hope this helps. If you have any comments or questions, leave a comment below.

This entry was posted in security and tagged , , , . Bookmark the permalink.

2 Responses to Configure SiLK on Linux for NetFlow collection from a Cisco router

  1. Nate Marks says:

    Thanks for the great article. I wonder if you’ve had any luck collecting netflow-v9 flows from a Cisco ASA? The configuration appears to be the same as the v5 config (with one obvious modification), but rwflowpack seems to just cough up errors.

    rwflowpack[14173]: fBufNext: NetFlow V9 template data record model type is not common with IPFIX IE model

    If this is as baffling to you as it is to me, do you know of any silk mailing lists? I found a bug report list, but I suspect this is really just a misconfiguration on my part.

    Thanks again!

  2. Nate, no problem, glad it was helpful. First question, did you build in support for “libfixbuf”? Per CERT: “This probe processes NetFlow v9 protocol data units (PDU) that the daemon reads from a UDP port from a router. To support NetFlow v9 probes, SiLK must be built with support for the external library libfixbuf, version 1.0 or later.”?

    For my v5 scenario I did not have to but for v9 and YaF it is required: http://www.rsreese.com/2011/08/configure-yaf-on-linux-for-netflow-collection-from-a-network-tap-or-span/ Let me know if that helps.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>