Parsing Netflow using Kibana via Logstash to ElasticSearch

This blog entry shows how to easily insert flow data into an ElasticSearch instance using Logstash and view the data using Kibana. To keep the example simple, we will use Kibana that is integrated in LogStash. We will not use the ElasticSearch that is bundled with LogStash. Instead, we will run latest stable version of ElasticSearch. Testing for this entry was done using Ubuntu 12.04 but most Linux or similar distributions should work fine.

First, I needed the ability to generate network flow. Softflowd provided a simple solution for my purposes. You skip the flow generation installation if you already have a v5 or v9 netflow source you could point to your LogStash instance. My testing was done with netflow version 9, but it appears the the LogStash netflow codec will also support 5. Softflowd required, byacc which you can get from here.

$ ./configure
$ make
$ sudo make install

Next, setup the netflow daemon that will create flow records from traffic on an interface that is designated. You can download the Softflowd source from here.

$ ./configure
$ make
$ sudo ./softflowd -i eth0 -n 127.0.0.1:12345 -v 9 -d

Before running ElasticSearch or LogStash, you will need Java. The latest 7.0 Java version should work just fine. You can confirm your Java version:

$ java -version

Before we run LogStash, grab the latest ElasticSearch version from the 0.90.x train. While ElasticSearch 1.x is out, I do not believe LogStash is yet compatible. If need be, you can edit the memory requirements in the following configuration file:

$ vim ./elasticsearch-0.90.12/bin/elasticsearch.in.sh

Next start the ElasticSearch instance:

$ sudo ./elasticsearch-0.90.12/bin/elasticsearch

Pull the latest LogStash JAR, before trying to run it, you will need a netflow configuration file. This configuration file says that we expect to receive network flow on UDP port 12345. Secondly, we output to STDOUT and the ElasticSearch entry, the former output is for testing.

input {
  udp {
    port => 12345
    codec => netflow
  }
}
output {
  stdout { }
  elasticsearch { host => "127.0.0.1" }
}

Next, we begin collecting netflow:

$ sudo java -jar ./Downloads/logstash-1.3.3-flatjar.jar agent -f logstash/netflow.conf -- &

After a minute or two, you should start seeing some entries via STDOUT in the terminal you started LogStash in. While you could start Kibana with the previous entry by adding the web toggle, I preferred separate instances for my evaluation:

$ sudo java -jar ./Downloads/logstash-1.3.3-flatjar.jar agent web -- &

Lastly, the fun part, you should be able to cruise over to either localhost or whatever IP address the systems as appending by port 9292 and starting tinkering:

http://127.0.0.1:9292

Here are three dashboards I quickly put together. Not only is Logstash a good way to quickly parse netflow, the dashboard shiny:

kibana1-thumb

kibana2-thumb

kibaba3-thumb

Leave a comment below if you have any questions.

Posted in network, software | Tagged , , | 2 Comments

Detecting Tor network traffic with YaF and Python

This entry continues a series of posts on identifying Tor network traffic and usage. The entry will demonstrate how to parse the output of YaF records via mediator using a Python script in order to determine if the SSL certificate values match the pattern of Tor certificates. It is assumed you have downloaded, compiled and installed YaF, mediator, and libfixbuf. Please see prior posts on this topic or the respective documentation for installation help if needed.

We first generate the YaF records from the PCAP we acquired. You can grab the example PCAP from cloudshark.

$ yaf --in tor.pcap --out tor.yaf

Next, parse the YaF output using mediator to disk in a format that we can parse. Alternatively, we could output to MySQL verse flat text files.

$ yaf_file_mediator-1.1.0/yaf_file_mediator --input tor.yaf --output tor.txt
**** Total flow count is 29 ****
**** Stats Total Count is 1 ****

Using Python, we can parse the records for patterns that match Tor SSL certificates.

#!/usr/bin/python

import re
import sys

filename = sys.argv[1]
myfile = open(filename,'r')
sourceIP = 'Source IP:'
destIP = 'Destination IP:'
issuerID = 'Issuer ID:'
subjectID = 'Subject ID:'
for line in myfile.readlines():
    line = line.strip()
    if line.startswith(sourceIP):
        sourceIPline = line
    elif line.startswith(destIP):
        destIPline = line
    elif line and line.startswith(issuerID):
        issuerDomain = re.search(r'www.\w+.com', line)
    elif line and line.startswith(subjectID):
        subjectDomain = re.search(r'www.\w+.net', line)
        if issuerDomain and subjectDomain:
            print (sourceIPline)
            print (destIPline)
            print issuerDomain.group()
            print subjectDomain.group()
            print
myfile.close

The following is an example output from the example PCAP provided earlier in this post. The Python regular expression ignores other SSL certificate values as they traditionally do not match the pattern that Tor certificates use, the inclusion of a domain for the Issuer and Subject ID’s. That said, false-positives could be introduced.

$ tor-ssl-parser.py tor.txt
Source IP: 10.0.0.126
Destination IP: 198.27.97.223
www.axslhtfqq.com
www.hkkch64skp7am.net

Source IP: 10.0.0.126
Destination IP: 96.127.153.58
www.rtqtkopfct767ai.com
www.facp2b2y5wjffbo5ioy.net

Source IP: 10.0.0.126
Destination IP: 192.151.147.5
www.5m6ywj2w7zs.com
www.iolbr3jbfs.net

Source IP: 10.0.0.126
Destination IP: 66.18.12.197
www.igdpzct5tauwgyqs.com
www.4tdznzbrfuv.net

Source IP: 10.0.0.126
Destination IP: 64.62.249.222
www.3pzqe4en5.com
www.glk3fwiz6.net

Source IP: 10.0.0.126
Destination IP: 212.83.158.173
www.lvv4l6sx3qafei2s5u.com
www.vznlngjz7a2fpg.net

Source IP: 10.0.0.126
Destination IP: 212.83.155.250
www.mbrdx4tz2ob5wlvazlr.com
www.shxl35n3zt.net

Source IP: 10.0.0.126
Destination IP: 212.83.140.45
www.3pxivyds.com
www.nolspqtib3ix.net

Source IP: 10.0.0.126
Destination IP: 212.83.158.50
www.s426lumoi7.com
www.ouzbot23a6lw3vvmszx.net

Source IP: 10.0.0.126
Destination IP: 212.83.158.40
www.3eexfeaw.com
www.iedhzej4tie4egm.net

Source IP: 10.0.0.126
Destination IP: 212.83.158.5
www.2fwld67ac2.com
www.6suxdq3miwwewq4.net

Source IP: 10.0.0.126
Destination IP: 31.7.186.228
www.5orbut4ufhohm5rlj47.com
www.orutxjqwf.net

Source IP: 10.0.0.126
Destination IP: 216.66.85.146
www.6pp7bfbdywvcaicqmfq.com
www.g6oa3qdobmdgl5tprm.net

Source IP: 10.0.0.126
Destination IP: 178.254.35.132
www.hbwpqbx4zimtptui.com
www.77wneeix55t.net

Source IP: 10.0.0.126
Destination IP: 188.40.98.96
www.ozsx22b4nda.com
www.lr7s5k3n6ber.net

Source IP: 10.0.0.126
Destination IP: 80.100.45.156
www.npmxal2ohuefme26yf.com
www.c7kriuquvh.net

Source IP: 10.0.0.126
Destination IP: 91.143.91.174
www.zcgg5yiwzajal4.com
www.55a4kx5jrqxezvk.net

Source IP: 10.0.0.126
Destination IP: 85.17.122.80
www.plgx26wgyroot37x3ysj.com
www.xwx5gpj5t2msq3.net

Source IP: 10.0.0.126
Destination IP: 88.159.20.120
www.s5rc22gpzrwt4e.com
www.qzsg2ioaoplbs2gaha5.net

Source IP: 10.0.0.126
Destination IP: 37.59.150.178
www.vywbff5wkza6npkd5l.com
www.ugdrrog5ro5wdfddj.net

Source IP: 10.0.0.126
Destination IP: 91.219.237.229
www.twngp3xrqgo4p.com
www.znskvp5k5pns22y2.net

Source IP: 10.0.0.126
Destination IP: 95.211.225.167
www.75ba5lymxpbhw3a2kb.com
www.rnspic4yus5crf6w.net

Source IP: 10.0.0.126
Destination IP: 82.96.35.7
www.spx5a4e5eyhkdtpt2xj.com
www.6phyovjhggkfm.net

Source IP: 10.0.0.126
Destination IP: 83.140.59.2
www.o5qzqtbs.com
www.bnymkm3nk7jtz3.net

Source IP: 10.0.0.126
Destination IP: 82.96.35.8
www.7wdf4rkj5mew.com
www.sd5mkmsmo.net

Source IP: 10.0.0.126
Destination IP: 93.180.156.45
www.rxy4jiw4wk.com
www.g66mipkcyhjwumywk4h.net

Source IP: 10.0.0.126
Destination IP: 81.218.109.195
www.gempmzrnwnk.com
www.6lrz7wtwprz.net

Source IP: 10.0.0.126
Destination IP: 31.172.30.4
www.4jvdpoo5wcklhd3usu.com
www.f4uxyorx2h.net

Source IP: 10.0.0.126
Destination IP: 50.7.194.122
www.pxznjv3t75.com
www.wuqq77l634eogfm.net

Please leave a comment if you have any questions.

Posted in network, security | Tagged , , , | Leave a comment

Detecting Tor network traffic with SiLK

This entry continues a series of posts on identifying Tor network traffic and usage. This post is not to argue the merits of allowing Tor to run on a network. However, the entry will demonstrate how to create a set of Tor server IP addresses to parse network flow using SiLK (System for Internet-Level Knowledge) in order to determine if the network flow is a match. It is assumed you have downloaded, compiled and installed SiLK, YaF, and libfixbuf. Please see prior posts on this topic or the respective documentation for installation help if needed.

We need to obtain the current list of Tor servers and place them in a file. We will then parse the destination IP addresses which will be placed into a SiLK set using the SiLK rwsetbuild command. Creating an IP set will allow us to use rwfilter to specify what IP addresses should match outgoing network traffic. A Perl script from here makes quick work of downloading the current Tor server list.

#!/usr/bin/perl
#
# Fetch the list of known Tor servers (from an existing Tor server) and
# display some of the basic info for each router.

use LWP::Simple;

# Hostname of an existing Tor router.  We use one of the directory authorities
# since that's pretty much what they're for.
$INITIAL_TOR_SERVER = "193.23.244.244";   # http://dannenberg.ccc.de/tor/status/all
$DIR_PORT = 80;

# Fetch the list of servers
$content = get("http://$INITIAL_TOR_SERVER:$DIR_PORT/tor/status/all");
@lines = split /\n/,$content;

foreach $router (@lines) {
    if($router =~ m/^r\s+(\S+)\s+(\S+)\s+(\S+)\s+(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\s+(\S+)\s+(\d+)\s+(\d+)$/) {
        ($name, $address, $or_port, $directory_port, $update_time) =
            ($1, $5, $6, $7, $4);
        print "$name | $address | $or_port | $directory_port | $update_time\n";
    }
}

Now that we have the current Tor server list, we can parse the Tor IP addresses. While you can modify the Perl script to only display the Tor server IP addresses, I still like to sort and parse for unique addresses as there are could be duplicates. You could also specify what type of Tor IP addresses you would like, i.e. exit, active, etc. Further, it is not bad to have a reference to determine what ports are associated with which addresses. Useful for more advanced queries.

$ awk -F "|" '{ print $2 }' exit-addresses | awk '{sub(/^[ \t]+/, "")};1' |sort|uniq > tor.txt

We convert the file containing the Tor server IP addresses to a set using the following command:

$ rwsetbuild tor.txt tor-servers.set

Typically, network flow would have already been captured for retrospective analysis, but for example sake, we will use a packet capture which already contains Tor traffic. We first convert our captured traffic to a YaF formatted file. This example PCAP may be downloaded from CloudShark.

$ /usr/local/bin/yaf --in tor.pcap --out ~/tor.yaf --filter="port 443" --applabel --applabel-rules=/usr/local/etc/yafApplabelRules.conf --max-payload=4000 --plugin-name=/usr/local/lib/yaf/dpacketplugin.la --plugin-opts="443" --lock &

Next, we convert the YaF format file to an IPFIX formatted file.

$ rwipfix2silk --silk-output=tor.rw tor.yaf

This rwfilter query parses for the data we are looking for and places in a binary file. We can write to standard out but I usually end up running additional queries using tools such as rwcut and rwstats so it is much faster to work from the smaller binary file, verse running the original query again.

$ rwfilter --start-date=2013/12/30 --end-date=2013/12/30  --dipset=tor-servers.set --proto=0- --type=all --pass=tor2.bin tor.rw

We parse the SiLK records we are interested in seeing to standard out via the rwcut command. Note the use of the cut command to minimize the white-space prefixing the output.

$ rwcut tor2.bin|cut -c26-
           sIP|                                    dIP|sPort|dPort|pro|   packets|     bytes|   flags|                  sTime| duration|                  eTime|sen|
    10.0.0.126|                          198.27.97.223|38946|  443|  6|        30|      8497|FS PA   |2013/12/30T20:20:21.336|   76.182|2013/12/30T20:21:37.518| S0|
 198.27.97.223|                             10.0.0.126|  443|38946|  6|        32|     28802|FS PA   |2013/12/30T20:20:21.381|   76.137|2013/12/30T20:21:37.518| S0|
    10.0.0.126|                          96.127.153.58|42529|  443|  6|        27|      8341|FS PA   |2013/12/30T20:20:22.190|   75.341|2013/12/30T20:21:37.531| S0|
 96.127.153.58|                             10.0.0.126|  443|42529|  6|        30|     26678|FS PA   |2013/12/30T20:20:22.232|   75.299|2013/12/30T20:21:37.531| S0|
    10.0.0.126|                          192.151.147.5|44384|  443|  6|        14|      3502|FS PA   |2013/12/30T20:20:26.486|   71.052|2013/12/30T20:21:37.538| S0|
 192.151.147.5|                             10.0.0.126|  443|44384|  6|        14|      4819|FS PA   |2013/12/30T20:20:26.535|   71.003|2013/12/30T20:21:37.538| S0|
    10.0.0.126|                           66.18.12.197|49341|  443|  6|        28|      8475|FS PA   |2013/12/30T20:20:21.426|   76.125|2013/12/30T20:21:37.551| S0|
  66.18.12.197|                             10.0.0.126|  443|49341|  6|        29|     26805|FS PA   |2013/12/30T20:20:21.471|   76.080|2013/12/30T20:21:37.551| S0|
    10.0.0.126|                          64.62.249.222|40742|  443|  6|        30|      8159|FS PA   |2013/12/30T20:20:21.375|   76.208|2013/12/30T20:21:37.583| S0|
 64.62.249.222|                             10.0.0.126|  443|40742|  6|        32|     28493|FS PA   |2013/12/30T20:20:21.461|   76.122|2013/12/30T20:21:37.583| S0|
    10.0.0.126|                         212.83.158.173|40825|  443|  6|        28|      8394|FS PA   |2013/12/30T20:20:22.079|   75.506|2013/12/30T20:21:37.585| S0|
212.83.158.173|                             10.0.0.126|  443|40825|  6|        31|     28867|FS PA   |2013/12/30T20:20:22.180|   75.405|2013/12/30T20:21:37.585| S0|
    10.0.0.126|                         212.83.155.250|55603|  443|  6|        29|      8454|FS PA   |2013/12/30T20:20:22.196|   75.389|2013/12/30T20:21:37.585| S0|
212.83.155.250|                             10.0.0.126|  443|55603|  6|        31|     27840|FS PA   |2013/12/30T20:20:22.290|   75.295|2013/12/30T20:21:37.585| S0|
    10.0.0.126|                          212.83.140.45|46797|  443|  6|        29|      8455|FS PA   |2013/12/30T20:20:21.342|   76.245|2013/12/30T20:21:37.587| S0|
 212.83.140.45|                             10.0.0.126|  443|46797|  6|        30|     26648|FS PA   |2013/12/30T20:20:21.439|   76.148|2013/12/30T20:21:37.587| S0|
    10.0.0.126|                          212.83.158.50|50935|  443|  6|        31|      8567|FS PA   |2013/12/30T20:20:21.396|   76.191|2013/12/30T20:21:37.587| S0|
 212.83.158.50|                             10.0.0.126|  443|50935|  6|        30|     26145|FS PA   |2013/12/30T20:20:21.492|   76.095|2013/12/30T20:21:37.587| S0|
    10.0.0.126|                          212.83.158.40|33170|  443|  6|        29|      8459|FS PA   |2013/12/30T20:20:22.088|   75.506|2013/12/30T20:21:37.594| S0|
 212.83.158.40|                             10.0.0.126|  443|33170|  6|        33|     28930|FS PA   |2013/12/30T20:20:23.199|   74.395|2013/12/30T20:21:37.594| S0|
    10.0.0.126|                           212.83.158.5|37960|  443|  6|        27|      8342|FS PA   |2013/12/30T20:20:21.415|   76.187|2013/12/30T20:21:37.602| S0|
  212.83.158.5|                             10.0.0.126|  443|37960|  6|        32|     26758|FS PA   |2013/12/30T20:20:21.517|   76.085|2013/12/30T20:21:37.602| S0|
    10.0.0.126|                           31.7.186.228|44997|  443|  6|        26|      8294|FS PA   |2013/12/30T20:20:21.377|   76.227|2013/12/30T20:21:37.604| S0|
  31.7.186.228|                             10.0.0.126|  443|44997|  6|        34|     29440|FS PA   |2013/12/30T20:20:21.486|   76.118|2013/12/30T20:21:37.604| S0|
    10.0.0.126|                          216.66.85.146|50817|  443|  6|        15|      3379|FS PA   |2013/12/30T20:21:34.492|    3.114|2013/12/30T20:21:37.606| S0|
 216.66.85.146|                             10.0.0.126|  443|50817|  6|        15|      6866|FS PA   |2013/12/30T20:21:34.590|    3.016|2013/12/30T20:21:37.606| S0|
    10.0.0.126|                         178.254.35.132|50724|  443|  6|        20|      5347|FS PA   |2013/12/30T20:20:33.494|   64.117|2013/12/30T20:21:37.611| S0|
178.254.35.132|                             10.0.0.126|  443|50724|  6|        23|     16358|FS PA   |2013/12/30T20:20:33.595|   64.016|2013/12/30T20:21:37.611| S0|
    10.0.0.126|                           188.40.98.96|54796|  443|  6|        30|      8565|FS PA   |2013/12/30T20:20:21.380|   76.231|2013/12/30T20:21:37.611| S0|
  188.40.98.96|                             10.0.0.126|  443|54796|  6|        32|     27966|FS PA   |2013/12/30T20:20:21.494|   76.117|2013/12/30T20:21:37.611| S0|
    10.0.0.126|                          80.100.45.156|60680|  443|  6|        30|      8578|FS PA   |2013/12/30T20:20:21.386|   76.228|2013/12/30T20:21:37.614| S0|
 80.100.45.156|                             10.0.0.126|  443|60680|  6|        31|     28447|FS PA   |2013/12/30T20:20:21.496|   76.118|2013/12/30T20:21:37.614| S0|
    10.0.0.126|                          91.143.91.174|39275|  443|  6|        23|      8209|FS PA   |2013/12/30T20:20:22.185|   75.435|2013/12/30T20:21:37.620| S0|
 91.143.91.174|                             10.0.0.126|  443|39275|  6|        33|     28626|FS PA   |2013/12/30T20:20:22.312|   75.308|2013/12/30T20:21:37.620| S0|
    10.0.0.126|                           85.17.122.80|43989|  443|  6|        29|      8457|FS PA   |2013/12/30T20:20:21.418|   76.202|2013/12/30T20:21:37.620| S0|
  85.17.122.80|                             10.0.0.126|  443|43989|  6|        32|     28409|FS PA   |2013/12/30T20:20:21.539|   76.081|2013/12/30T20:21:37.620| S0|
    10.0.0.126|                          88.159.20.120|49609|  443|  6|        31|      8633|FS PA   |2013/12/30T20:20:21.412|   76.208|2013/12/30T20:21:37.620| S0|
 88.159.20.120|                             10.0.0.126|  443|49609|  6|        34|     29194|FS PA   |2013/12/30T20:20:21.513|   76.107|2013/12/30T20:21:37.620| S0|
    10.0.0.126|                          37.59.150.178|47658|  443|  6|        30|      8516|FS PA   |2013/12/30T20:20:21.399|   76.223|2013/12/30T20:21:37.622| S0|
 37.59.150.178|                             10.0.0.126|  443|47658|  6|        33|     29412|FS PA   |2013/12/30T20:20:21.513|   76.109|2013/12/30T20:21:37.622| S0|
    10.0.0.126|                         91.219.237.229|35498|  443|  6|        15|      3616|FS PA   |2013/12/30T20:21:34.489|    3.134|2013/12/30T20:21:37.623| S0|
91.219.237.229|                             10.0.0.126|  443|35498|  6|        14|      7664|FS PA   |2013/12/30T20:21:34.614|    3.009|2013/12/30T20:21:37.623| S0|
    10.0.0.126|                         95.211.225.167|57656|  443|  6|        27|      8359|FS PA   |2013/12/30T20:20:21.345|   76.280|2013/12/30T20:21:37.625| S0|
95.211.225.167|                             10.0.0.126|  443|57656|  6|        33|     27948|FS PA   |2013/12/30T20:20:21.475|   76.150|2013/12/30T20:21:37.625| S0|
    10.0.0.126|                             82.96.35.7|58655|  443|  6|        15|      3563|FS PA   |2013/12/30T20:21:34.486|    3.147|2013/12/30T20:21:37.633| S0|
    82.96.35.7|                             10.0.0.126|  443|58655|  6|        13|      7445|FS PA   |2013/12/30T20:21:34.629|    3.004|2013/12/30T20:21:37.633| S0|
    10.0.0.126|                            83.140.59.2|45720|  443|  6|        22|      8160|FS PA   |2013/12/30T20:20:21.745|   75.888|2013/12/30T20:21:37.633| S0|
   83.140.59.2|                             10.0.0.126|  443|45720|  6|        30|     27422|FS PA   |2013/12/30T20:20:21.887|   75.746|2013/12/30T20:21:37.633| S0|
    10.0.0.126|                             82.96.35.8|42995|  443|  6|        28|      8414|FS PA   |2013/12/30T20:20:21.339|   76.302|2013/12/30T20:21:37.641| S0|
    82.96.35.8|                             10.0.0.126|  443|42995|  6|        33|     28927|FS PA   |2013/12/30T20:20:21.479|   76.162|2013/12/30T20:21:37.641| S0|
    10.0.0.126|                          93.180.156.45|47282|  443|  6|        33|      8671|FS PA   |2013/12/30T20:20:21.421|   76.223|2013/12/30T20:21:37.644| S0|
 93.180.156.45|                             10.0.0.126|  443|47282|  6|        39|     31370|FS PA   |2013/12/30T20:20:21.562|   76.082|2013/12/30T20:21:37.644| S0|
    10.0.0.126|                         81.218.109.195|60000|  443|  6|        29|      8460|FS PA   |2013/12/30T20:20:21.383|   76.277|2013/12/30T20:21:37.660| S0|
81.218.109.195|                             10.0.0.126|  443|60000|  6|        32|     27852|FS PA   |2013/12/30T20:20:21.535|   76.125|2013/12/30T20:21:37.660| S0|
    10.0.0.126|                            31.172.30.4|35914|  443|  6|        36|      8922|FS PA   |2013/12/30T20:20:22.146|   75.538|2013/12/30T20:21:37.684| S0|
   31.172.30.4|                             10.0.0.126|  443|35914|  6|        34|     32082|FS PA   |2013/12/30T20:20:22.271|   75.413|2013/12/30T20:21:37.684| S0|
    10.0.0.126|                           50.7.194.122|38522|  443|  6|        20|      5384|FS PA   |2013/12/30T20:20:33.487|   64.202|2013/12/30T20:21:37.689| S0|
  50.7.194.122|                             10.0.0.126|  443|38522|  6|        17|      9223|FS PA   |2013/12/30T20:20:33.671|   64.018|2013/12/30T20:21:37.689| S0|

With the next query, we adjust the type of traffic we want to look at to only outgoing traffic to the Tor servers instead of the previously displayed bi-directional traffic.

$ rwfilter --dipset=tor-servers.set --proto=0- --type=out --pass=tor.bin tor.rw

Again, we parse the SiLK records. Again, note the use of the cut command to minimize the white-space prefix the first column of data. The reason for this is there are additional columns of data not displayed by default. Checkout the rwcut man page for other columns data that may be of interest.

$ rwcut tor.bin |cut -c30-
       sIP|                                    dIP|sPort|dPort|pro|   packets|     bytes|   flags|                  sTime| duration|                  eTime|sen|
10.0.0.126|                          198.27.97.223|38946|  443|  6|        30|      8497|FS PA   |2013/12/30T20:20:21.336|   76.182|2013/12/30T20:21:37.518| S0|
10.0.0.126|                          96.127.153.58|42529|  443|  6|        27|      8341|FS PA   |2013/12/30T20:20:22.190|   75.341|2013/12/30T20:21:37.531| S0|
10.0.0.126|                          192.151.147.5|44384|  443|  6|        14|      3502|FS PA   |2013/12/30T20:20:26.486|   71.052|2013/12/30T20:21:37.538| S0|
10.0.0.126|                           66.18.12.197|49341|  443|  6|        28|      8475|FS PA   |2013/12/30T20:20:21.426|   76.125|2013/12/30T20:21:37.551| S0|
10.0.0.126|                          64.62.249.222|40742|  443|  6|        30|      8159|FS PA   |2013/12/30T20:20:21.375|   76.208|2013/12/30T20:21:37.583| S0|
10.0.0.126|                         212.83.158.173|40825|  443|  6|        28|      8394|FS PA   |2013/12/30T20:20:22.079|   75.506|2013/12/30T20:21:37.585| S0|
10.0.0.126|                         212.83.155.250|55603|  443|  6|        29|      8454|FS PA   |2013/12/30T20:20:22.196|   75.389|2013/12/30T20:21:37.585| S0|
10.0.0.126|                          212.83.140.45|46797|  443|  6|        29|      8455|FS PA   |2013/12/30T20:20:21.342|   76.245|2013/12/30T20:21:37.587| S0|
10.0.0.126|                          212.83.158.50|50935|  443|  6|        31|      8567|FS PA   |2013/12/30T20:20:21.396|   76.191|2013/12/30T20:21:37.587| S0|
10.0.0.126|                          212.83.158.40|33170|  443|  6|        29|      8459|FS PA   |2013/12/30T20:20:22.088|   75.506|2013/12/30T20:21:37.594| S0|
10.0.0.126|                           212.83.158.5|37960|  443|  6|        27|      8342|FS PA   |2013/12/30T20:20:21.415|   76.187|2013/12/30T20:21:37.602| S0|
10.0.0.126|                           31.7.186.228|44997|  443|  6|        26|      8294|FS PA   |2013/12/30T20:20:21.377|   76.227|2013/12/30T20:21:37.604| S0|
10.0.0.126|                          216.66.85.146|50817|  443|  6|        15|      3379|FS PA   |2013/12/30T20:21:34.492|    3.114|2013/12/30T20:21:37.606| S0|
10.0.0.126|                         178.254.35.132|50724|  443|  6|        20|      5347|FS PA   |2013/12/30T20:20:33.494|   64.117|2013/12/30T20:21:37.611| S0|
10.0.0.126|                           188.40.98.96|54796|  443|  6|        30|      8565|FS PA   |2013/12/30T20:20:21.380|   76.231|2013/12/30T20:21:37.611| S0|
10.0.0.126|                          80.100.45.156|60680|  443|  6|        30|      8578|FS PA   |2013/12/30T20:20:21.386|   76.228|2013/12/30T20:21:37.614| S0|
10.0.0.126|                          91.143.91.174|39275|  443|  6|        23|      8209|FS PA   |2013/12/30T20:20:22.185|   75.435|2013/12/30T20:21:37.620| S0|
10.0.0.126|                           85.17.122.80|43989|  443|  6|        29|      8457|FS PA   |2013/12/30T20:20:21.418|   76.202|2013/12/30T20:21:37.620| S0|
10.0.0.126|                          88.159.20.120|49609|  443|  6|        31|      8633|FS PA   |2013/12/30T20:20:21.412|   76.208|2013/12/30T20:21:37.620| S0|
10.0.0.126|                          37.59.150.178|47658|  443|  6|        30|      8516|FS PA   |2013/12/30T20:20:21.399|   76.223|2013/12/30T20:21:37.622| S0|
10.0.0.126|                         91.219.237.229|35498|  443|  6|        15|      3616|FS PA   |2013/12/30T20:21:34.489|    3.134|2013/12/30T20:21:37.623| S0|
10.0.0.126|                         95.211.225.167|57656|  443|  6|        27|      8359|FS PA   |2013/12/30T20:20:21.345|   76.280|2013/12/30T20:21:37.625| S0|
10.0.0.126|                             82.96.35.7|58655|  443|  6|        15|      3563|FS PA   |2013/12/30T20:21:34.486|    3.147|2013/12/30T20:21:37.633| S0|
10.0.0.126|                            83.140.59.2|45720|  443|  6|        22|      8160|FS PA   |2013/12/30T20:20:21.745|   75.888|2013/12/30T20:21:37.633| S0|
10.0.0.126|                             82.96.35.8|42995|  443|  6|        28|      8414|FS PA   |2013/12/30T20:20:21.339|   76.302|2013/12/30T20:21:37.641| S0|
10.0.0.126|                          93.180.156.45|47282|  443|  6|        33|      8671|FS PA   |2013/12/30T20:20:21.421|   76.223|2013/12/30T20:21:37.644| S0|
10.0.0.126|                         81.218.109.195|60000|  443|  6|        29|      8460|FS PA   |2013/12/30T20:20:21.383|   76.277|2013/12/30T20:21:37.660| S0|
10.0.0.126|                            31.172.30.4|35914|  443|  6|        36|      8922|FS PA   |2013/12/30T20:20:22.146|   75.538|2013/12/30T20:21:37.684| S0|
10.0.0.126|                           50.7.194.122|38522|  443|  6|        20|      5384|FS PA   |2013/12/30T20:20:33.487|   64.202|2013/12/30T20:21:37.689| S0|

Lastly, we take a look at the reverse entries. As you can see, it is apparent that some of the hosts have Tor tertiary domain names which suggests that some of the flows may be destined for Tor servers.

$ rwcut tor.bin |rwresolve |cut -c30-
       sIP|                                    dIP|sPort|dPort|pro|   packets|     bytes|   flags|                  sTime| duration|                  eTime|sen|
10.0.0.126|198.27.97.223.vpsrealm.com|38946|  443|  6|        30|      8497|FS PA   |2013/12/30T20:20:21.336|   76.182|2013/12/30T20:21:37.518| S0|
10.0.0.126|xxviii.example.tld|42529|  443|  6|        27|      8341|FS PA   |2013/12/30T20:20:22.190|   75.341|2013/12/30T20:21:37.531| S0|
10.0.0.126|tor.koehn.com|44384|  443|  6|        14|      3502|FS PA   |2013/12/30T20:20:26.486|   71.052|2013/12/30T20:21:37.538| S0|
10.0.0.126|                           66.18.12.197|49341|  443|  6|        28|      8475|FS PA   |2013/12/30T20:20:21.426|   76.125|2013/12/30T20:21:37.551| S0|
10.0.0.126|hecustomer.10gigabitethernet8-1.core1.pao1.he.net|40742|  443|  6|        30|      8159|FS PA   |2013/12/30T20:20:21.375|   76.208|2013/12/30T20:21:37.583| S0|
10.0.0.126|n5.servbr.net|40825|  443|  6|        28|      8394|FS PA   |2013/12/30T20:20:22.079|   75.506|2013/12/30T20:21:37.585| S0|
10.0.0.126|n15.servbr.net|55603|  443|  6|        29|      8454|FS PA   |2013/12/30T20:20:22.196|   75.389|2013/12/30T20:21:37.585| S0|
10.0.0.126|212-83-140-45.rev.poneytelecom.eu|46797|  443|  6|        29|      8455|FS PA   |2013/12/30T20:20:21.342|   76.245|2013/12/30T20:21:37.587| S0|
10.0.0.126|n13.servbr.net|50935|  443|  6|        31|      8567|FS PA   |2013/12/30T20:20:21.396|   76.191|2013/12/30T20:21:37.587| S0|
10.0.0.126|n12.servbr.net|33170|  443|  6|        29|      8459|FS PA   |2013/12/30T20:20:22.088|   75.506|2013/12/30T20:21:37.594| S0|
10.0.0.126|n10.servbr.net|37960|  443|  6|        27|      8342|FS PA   |2013/12/30T20:20:21.415|   76.187|2013/12/30T20:21:37.602| S0|
10.0.0.126|                           31.7.186.228|44997|  443|  6|        26|      8294|FS PA   |2013/12/30T20:20:21.377|   76.227|2013/12/30T20:21:37.604| S0|
10.0.0.126|hecustomer.10gigabitethernet1-2.core1.ams1.he.net|50817|  443|  6|        15|      3379|FS PA   |2013/12/30T20:21:34.492|    3.114|2013/12/30T20:21:37.606| S0|
10.0.0.126|v37433.1blu.de|50724|  443|  6|        20|      5347|FS PA   |2013/12/30T20:20:33.494|   64.117|2013/12/30T20:21:37.611| S0|
10.0.0.126|static.188-40-98-96.clients.your-server.de|54796|  443|  6|        30|      8565|FS PA   |2013/12/30T20:20:21.380|   76.231|2013/12/30T20:21:37.611| S0|
10.0.0.126|a80-100-45-156.adsl.xs4all.nl|60680|  443|  6|        30|      8578|FS PA   |2013/12/30T20:20:21.386|   76.228|2013/12/30T20:21:37.614| S0|
10.0.0.126|91.143.91.174|39275|  443|  6|        23|      8209|FS PA   |2013/12/30T20:20:22.185|   75.435|2013/12/30T20:21:37.620| S0|
10.0.0.126|                           85.17.122.80|43989|  443|  6|        29|      8457|FS PA   |2013/12/30T20:20:21.418|   76.202|2013/12/30T20:21:37.620| S0|
10.0.0.126|120-20-159-88.business.edutel.nl|49609|  443|  6|        31|      8633|FS PA   |2013/12/30T20:20:21.412|   76.208|2013/12/30T20:21:37.620| S0|
10.0.0.126|37-59-150-178.static-ip.hostplanet.me|47658|  443|  6|        30|      8516|FS PA   |2013/12/30T20:20:21.399|   76.223|2013/12/30T20:21:37.622| S0|
10.0.0.126|sa0111.azar-a.net|35498|  443|  6|        15|      3616|FS PA   |2013/12/30T20:21:34.489|    3.134|2013/12/30T20:21:37.623| S0|
10.0.0.126|greendale.badexample.net|57656|  443|  6|        27|      8359|FS PA   |2013/12/30T20:20:21.345|   76.280|2013/12/30T20:21:37.625| S0|
10.0.0.126|luftgitarr.mooo.se|58655|  443|  6|        15|      3563|FS PA   |2013/12/30T20:21:34.486|    3.147|2013/12/30T20:21:37.633| S0|
10.0.0.126|kimya.mooo.se|45720|  443|  6|        22|      8160|FS PA   |2013/12/30T20:20:21.745|   75.888|2013/12/30T20:21:37.633| S0|
10.0.0.126|junis.mooo.se|42995|  443|  6|        28|      8414|FS PA   |2013/12/30T20:20:21.339|   76.302|2013/12/30T20:21:37.641| S0|
10.0.0.126|tor.b0red.de|47282|  443|  6|        33|      8671|FS PA   |2013/12/30T20:20:21.421|   76.223|2013/12/30T20:21:37.644| S0|
10.0.0.126|195.ab4.interhost.co.il|60000|  443|  6|        29|      8460|FS PA   |2013/12/30T20:20:21.383|   76.277|2013/12/30T20:21:37.660| S0|
10.0.0.126|tor21.anonymizer.ccc.de|35914|  443|  6|        36|      8922|FS PA   |2013/12/30T20:20:22.146|   75.538|2013/12/30T20:21:37.684| S0|
10.0.0.126|torsrvl.snydernet.net|38522|  443|  6|        20|      5384|FS PA   |2013/12/30T20:20:33.487|   64.202|2013/12/30T20:21:37.689| S0|

Or we can use the rwuniq command to list the unique destinations, again piping through rwresolve:

$ rwuniq --fields=2 --no-columns tor.bin |rwresolve
dIP|Records|
luftgitarr.mooo.se|1|
tor.b0red.de|1|
junis.mooo.se|1|
31.7.186.228|1|
tor21.anonymizer.ccc.de|1|
xxviii.example.tld|1|
tor.koehn.com|1|
n15.servbr.net|1|
a80-100-45-156.adsl.xs4all.nl|1|
n13.servbr.net|1|
120-20-159-88.business.edutel.nl|1|
91.143.91.174|1|
195.ab4.interhost.co.il|1|
37-59-150-178.static-ip.hostplanet.me|1|
sa0111.azar-a.net|1|
static.188-40-98-96.clients.your-server.de|1|
n5.servbr.net|1|
torsrvl.snydernet.net|1|
198.27.97.223.vpsrealm.com|1|
66.18.12.197|1|
v37433.1blu.de|1|
hecustomer.10gigabitethernet1-2.core1.ams1.he.net|1|
212-83-140-45.rev.poneytelecom.eu|1|
kimya.mooo.se|1|
85.17.122.80|1|
n12.servbr.net|1|
greendale.badexample.net|1|
n10.servbr.net|1|
hecustomer.10gigabitethernet8-1.core1.pao1.he.net|1|

In conclusion, using SiLK we can provide retrospective analysis to determine if traffic may be destined for Tor servers. While not a definitive method of detection as there could be false-positives due to hosting of legitimate services on Tor servers, it is a quick method to get some insight. As usual, please leave a comment below if you have any questions or comments.

Posted in network, security | Tagged , , | Leave a comment

Detecting Tor traffic with Bro network traffic analyzer

This entry is a post in a series to identifying Tor (the onion router) network traffic and usage using the Bro network traffic analyzer. To learn more about both projects, please visit the aforementioned links. This post is not to argue the merits of allowing Tor to run on a network. Due to malware variants taking advantage of Tor for it’s botnet command and control (C2), I wanted to be able to effectively identify Tor usage in hopes of identifying hosts that may be using Tor for C2 purposes.

A method folks often use to identify communication with Tor relays is to compare the current list of known Tor servers with the traffic from their network. While this does work, some relay’s may host other legitimate services which could introduce false-positives. The goal was to find a method to augment the parsing network traffic for Tor server matches which is sometimes done retrospectively.

If we take a look at the Tor certificates, we see an interesting pattern for the Issuer and Subject ID form a pattern.

Using tshark, it the Issuer and Subject ID patterns are a little more apparent.

$ tshark -r tor.pcap -T fields -R "ssl.handshake.certificate" -e x509af.utcTime -e x509sat.printableString
13-10-15 00:00:00 (UTC),14-02-11 23:59:59 (UTC)	www.axslhtfqq.com,www.hkkch64skp7am.net
13-12-30 18:32:48 (UTC),14-12-30 18:32:48 (UTC)	www.igdpzct5tauwgyqs.com,www.4tdznzbrfuv.net
13-10-04 00:00:00 (UTC),14-04-22 00:00:00 (UTC)	www.3pxivyds.com,www.nolspqtib3ix.net
13-11-17 00:00:00 (UTC),14-06-22 00:00:00 (UTC)	www.3pzqe4en5.com,www.glk3fwiz6.net
13-06-19 00:00:00 (UTC),14-04-20 00:00:00 (UTC)	www.5orbut4ufhohm5rlj47.com,www.orutxjqwf.net
13-06-15 00:00:00 (UTC),14-02-04 00:00:00 (UTC)	www.7wdf4rkj5mew.com,www.sd5mkmsmo.net
13-11-19 00:00:00 (UTC),14-02-05 23:59:59 (UTC)	www.75ba5lymxpbhw3a2kb.com,www.rnspic4yus5crf6w.net
13-12-30 19:54:02 (UTC),14-12-30 19:54:02 (UTC)	www.s5rc22gpzrwt4e.com,www.qzsg2ioaoplbs2gaha5.net
13-08-12 00:00:00 (UTC),14-04-16 23:59:59 (UTC)	www.2fwld67ac2.com,www.6suxdq3miwwewq4.net
13-12-18 00:00:00 (UTC),14-02-14 23:59:59 (UTC)	www.npmxal2ohuefme26yf.com,www.c7kriuquvh.net
13-10-18 00:00:00 (UTC),14-06-16 00:00:00 (UTC)	www.s426lumoi7.com,www.ouzbot23a6lw3vvmszx.net
13-12-31 00:00:00 (UTC),14-02-01 23:59:59 (UTC)	www.vywbff5wkza6npkd5l.com,www.ugdrrog5ro5wdfddj.net
13-11-27 00:00:00 (UTC),14-08-13 00:00:00 (UTC)	www.ozsx22b4nda.com,www.lr7s5k3n6ber.net
13-03-31 00:00:00 (UTC),14-01-06 23:59:59 (UTC)	www.plgx26wgyroot37x3ysj.com,www.xwx5gpj5t2msq3.net
13-12-18 00:00:00 (UTC),14-02-20 00:00:00 (UTC)	www.gempmzrnwnk.com,www.6lrz7wtwprz.net
13-08-16 00:00:00 (UTC),14-01-26 23:59:59 (UTC)	www.rxy4jiw4wk.com,www.g66mipkcyhjwumywk4h.net
13-12-30 19:07:41 (UTC),14-12-30 19:07:41 (UTC)	www.o5qzqtbs.com,www.bnymkm3nk7jtz3.net
13-07-27 00:00:00 (UTC),14-01-18 00:00:00 (UTC)	www.rtqtkopfct767ai.com,www.facp2b2y5wjffbo5ioy.net
13-09-09 00:00:00 (UTC),14-02-26 00:00:00 (UTC)	www.lvv4l6sx3qafei2s5u.com,www.vznlngjz7a2fpg.net
13-12-21 00:00:00 (UTC),14-02-08 00:00:00 (UTC)	www.mbrdx4tz2ob5wlvazlr.com,www.shxl35n3zt.net
13-12-12 00:00:00 (UTC),14-01-15 00:00:00 (UTC)	www.4jvdpoo5wcklhd3usu.com,www.f4uxyorx2h.net
13-10-17 00:00:00 (UTC),14-05-05 00:00:00 (UTC)	www.zcgg5yiwzajal4.com,www.55a4kx5jrqxezvk.net
13-05-18 00:00:00 (UTC),14-04-07 23:59:59 (UTC)	www.3eexfeaw.com,www.iedhzej4tie4egm.net
13-12-23 00:00:00 (UTC),14-01-22 23:59:59 (UTC)	www.5m6ywj2w7zs.com,www.iolbr3jbfs.net
13-03-09 00:00:00 (UTC),14-01-01 23:59:59 (UTC)	www.hbwpqbx4zimtptui.com,www.77wneeix55t.net
13-12-26 00:00:00 (UTC),14-04-19 00:00:00 (UTC)	www.pxznjv3t75.com,www.wuqq77l634eogfm.net
13-12-07 00:00:00 (UTC),14-03-17 23:59:59 (UTC)	www.6pp7bfbdywvcaicqmfq.com,www.g6oa3qdobmdgl5tprm.net
13-12-30 19:42:49 (UTC),14-12-30 19:42:49 (UTC)	www.twngp3xrqgo4p.com,www.znskvp5k5pns22y2.net
13-02-14 00:00:00 (UTC),14-01-14 00:00:00 (UTC)	www.spx5a4e5eyhkdtpt2xj.com,www.6phyovjhggkfm.net

So with this knowledge I started looking to see if there were any current methods of identifying the anomalous certificate identifiers. Lucky for Bro users, Seth Hall created a detect-tor.bro script to do just that. I downloaded the latest Bro 2.2 source package and built it on my Ubuntu VM. I also pulled down the aforementioned detect-tor.bro script.

My first attempt did not go well. I was greeted with a warning and did not see the logs I had hoped to see.

$ sudo /usr/local/bro/bin/bro -r tor.pcap detect-tor.bro
1388434831.619531 warning in /usr/local/bro/share/bro/base/misc/find-checksum-offloading.bro, line 54: Your trace file likely has invalid TCP checksums, most likely from NIC checksum offloading.

This was quickly fixed by including the -C toggle to ignore checksums.

$ sudo /usr/local/bro/bin/bro -C -r tor.pcap detect-tor.bro

After parsing the Tor traffic collected (which you can pull down from CloudShark if you do not feel like generating your own), we have some interesting logs. At first glace, we see an alert from the detect-tor.bro script. While the event is pretty self explanatory, note the destination IP addresses are not included because Tor will usually have multiple servers, i.e. destination IP addresses.

$ more notice.log
#separator \x09
#set_separator  ,
#empty_field    (empty)
#unset_field    -
#path   notice
#open   2014-01-03-14-12-05
#fields ts      uid     id.orig_h       id.orig_p       id.resp_h       id.resp_p       fuid    file_mime_type  file_desc       proto   note    msg     sub     src  dst      p       n       peer_descr      actions suppress_for    dropped remote_location.country_code    remote_location.region  remote_location.city    remote_locatio
n.latitude      remote_location.longitude
#types  time    string  addr    port    addr    port    string  string  string  enum    enum    string  string  addr    addr    port    count   string  table[enum]  interval bool    string  string  string  double  double
1388434821.597322       -       -       -       -       -       -       -       -       -       DetectTor::Found        10.0.0.126 was found using Tor by connecting t
o servers with at least 10 unique weird certs   -       10.0.0.126      -       -       -       bro     Notice::ACTION_LOG      3600.000000     F       -       -    --       -
#close  2014-01-03-14-12-05

We can cut down entry noise by specifying only what we want to see:

$ cat notice.log|/usr/local/bro/bin/bro-cut -c -d note msg src dst actions suppress_for dropped
#separator \x09
#set_separator  ,
#empty_field    (empty)
#unset_field    -
#path   notice
#open   2014-01-03-14-12-05
#fields note    msg     src     dst     actions suppress_for    dropped
#types  string  string  addr    addr    table[enum]     interval        bool
DetectTor::Found        10.0.0.126 was found using Tor by connecting to servers with at least 10 unique weird certs     10.0.0.126      -       Notice::ACTION_LOG   3600.000000      F

After seeing the alert in the notice.log, we can take a look in the ssl.log to determine what traffic caused the alert to fire.

$ more ssl.log
#separator \x09
#set_separator  ,
#empty_field    (empty)
#unset_field    -
#path   ssl
#open   2014-01-03-14-12-05
#fields ts      uid     id.orig_h       id.orig_p       id.resp_h       id.resp_p       version cipher  server_name     session_id      subject issuer_subject  not_va
lid_before      not_valid_after last_alert      client_subject  client_issuer_subject
#types  time    string  addr    port    addr    port    string  string  string  string  string  string  time    time    string  string  string
1388434821.514935       CwRHlF31djcMrO7Z98      10.0.0.126      51191   199.36.221.196  9001    TLSv10  TLS_DHE_RSA_WITH_AES_256_CBC_SHA        www.wplgkqpnteb.com  -CN=www.ri6ufvqioii5se5tzbgt.net CN=www.dyyp6enzivlm46.com       1388447336.000000       1419983336.000000       -       -       -
1388434821.482053       Ck1Mgy4ubChMFyneFc      10.0.0.126      38946   198.27.97.223   443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.p65b.com    -    CN=www.hkkch64skp7am.net CN=www.axslhtfqq.com    1381809600.000000       1392181199.000000       -       -       -
1388434821.533291       CZOEio3mxlQgpmVD2i      10.0.0.126      36715   149.9.0.60      9001    TLSv10  TLS_DHE_RSA_WITH_AES_256_CBC_SHA        www.dpvdl3n6yzwv.com -CN=www.anojueopqlpgsj.net       CN=www.u2rsltgpogir6t.com       1384405200.000000       1398830399.000000       -       -       -
1388434821.484476       CnU0VyJcJHaeCaxh8       10.0.0.126      49341   66.18.12.197    443     TLSv10  TLS_DHE_RSA_WITH_AES_256_CBC_SHA        www.6kyx72vjlrwxcmxnj4
we7n.com        -       CN=www.4tdznzbrfuv.net  CN=www.igdpzct5tauwgyqs.com     1388446368.000000       1419982368.000000       -       -       -
1388434821.484255       Cc00yR3kKWb2GstwXf      10.0.0.126      40742   64.62.249.222   443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.de5v2whiex3xxy.com
        -       CN=www.glk3fwiz6.net    CN=www.3pzqe4en5.com    1384664400.000000       1403409600.000000       -       -       -
1388434821.583284       CuVFNK14saFKjGVhfh      10.0.0.126      54393   50.115.122.68   9001    TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.ojj4rbje7z7.com  -CN=www.qexiojanju56.net CN=www.nnfslkrseh.com   1387342800.000000       1390280400.000000       -       -       -
1388434821.482585       CROLl5Vd0jUzvvwn        10.0.0.126      46797   212.83.140.45   443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.esd7jqvwpbwebf.com
        -       CN=www.nolspqtib3ix.net CN=www.3pxivyds.com     1380859200.000000       1398139200.000000       -       -       -
1388434821.597288       CXemGQ4G0PFf5DvUf       10.0.0.126      34887   72.52.91.30     5901    TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.igyewbs5.com     -CN=www.bnlln35al.net    CN=www.henq76fjat2ozl2537.com   1376020800.000000       1403841600.000000       -       -       -
1388434821.597322       CFrNiH22BOLl917zjl      10.0.0.126      56135   144.76.109.178  9081    TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.57xl.com    -    CN=www.3rvuayihf4t35h.net        CN=www.viw7rvktu36ov.com        1386651600.000000       1388811600.000000       -       -       -
1388434821.489984       CxEp7Xmn9AOlkxn0e       10.0.0.126      44997   31.7.186.228    443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.ewrk2xtmr.com    -CN=www.orutxjqwf.net    CN=www.5orbut4ufhohm5rlj47.com  1371614400.000000       1397966400.000000       -       -       -

Again, we can select the fields we want to see in order to minimize output.

$ cat ssl.log|/usr/local/bro/bin/bro-cut -c -d ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher server_name subject issuer_subject not_valid_before not_valid_after
#separator \x09
#set_separator  ,
#empty_field    (empty)
#unset_field    -
#path   ssl
#open   2014-01-03-14-12-05
#fields ts      uid     id.orig_h       id.orig_p       id.resp_h       id.resp_p       version cipher  server_name     subject issuer_subject  not_valid_before     not_valid_after
#types  string  string  addr    port    addr    port    string  string  string  string  string  time    string
2013-12-30T15:20:21-0500        CwRHlF31djcMrO7Z98      10.0.0.126      51191   199.36.221.196  9001    TLSv10  TLS_DHE_RSA_WITH_AES_256_CBC_SHA        www.wplgkqpnteb.com   CN=www.ri6ufvqioii5se5tzbgt.net CN=www.dyyp6enzivlm46.com       2013-12-30T18:48:56-0500        2014-12-30T18:48:56-0500
2013-12-30T15:20:21-0500        Ck1Mgy4ubChMFyneFc      10.0.0.126      38946   198.27.97.223   443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.p65b.com CN=www.hkkch64skp7am.net CN=www.axslhtfqq.com    2013-10-15T00:00:00-0400        2014-02-11T23:59:59-0500
2013-12-30T15:20:21-0500        CZOEio3mxlQgpmVD2i      10.0.0.126      36715   149.9.0.60      9001    TLSv10  TLS_DHE_RSA_WITH_AES_256_CBC_SHA        www.dpvdl3n6yzwv.com  CN=www.anojueopqlpgsj.net       CN=www.u2rsltgpogir6t.com       2013-11-14T00:00:00-0500        2014-04-29T23:59:59-0400
2013-12-30T15:20:21-0500        CnU0VyJcJHaeCaxh8       10.0.0.126      49341   66.18.12.197    443     TLSv10  TLS_DHE_RSA_WITH_AES_256_CBC_SHA        www.6kyx72vjlrwxcmxnj4we7n.com        CN=www.4tdznzbrfuv.net  CN=www.igdpzct5tauwgyqs.com     2013-12-30T18:32:48-0500        2014-12-30T18:32:48-0500
2013-12-30T15:20:21-0500        Cc00yR3kKWb2GstwXf      10.0.0.126      40742   64.62.249.222   443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.de5v2whiex3xxy.com        CN=www.glk3fwiz6.net    CN=www.3pzqe4en5.com    2013-11-17T00:00:00-0500        2014-06-22T00:00:00-0400
2013-12-30T15:20:21-0500        CuVFNK14saFKjGVhfh      10.0.0.126      54393   50.115.122.68   9001    TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.ojj4rbje7z7.com   CN=www.qexiojanju56.net CN=www.nnfslkrseh.com   2013-12-18T00:00:00-0500        2014-01-21T00:00:00-0500
2013-12-30T15:20:21-0500        CROLl5Vd0jUzvvwn        10.0.0.126      46797   212.83.140.45   443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.esd7jqvwpbwebf.com        CN=www.nolspqtib3ix.net CN=www.3pxivyds.com     2013-10-04T00:00:00-0400        2014-04-22T00:00:00-0400
2013-12-30T15:20:21-0500        CXemGQ4G0PFf5DvUf       10.0.0.126      34887   72.52.91.30     5901    TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.igyewbs5.com      CN=www.bnlln35al.net    CN=www.henq76fjat2ozl2537.com   2013-08-09T00:00:00-0400        2014-06-27T00:00:00-0400
2013-12-30T15:20:21-0500        CFrNiH22BOLl917zjl      10.0.0.126      56135   144.76.109.178  9081    TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.57xl.com CN=www.3rvuayihf4t35h.net        CN=www.viw7rvktu36ov.com        2013-12-10T00:00:00-0500        2014-01-04T00:00:00-0500
2013-12-30T15:20:21-0500        CxEp7Xmn9AOlkxn0e       10.0.0.126      44997   31.7.186.228    443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.ewrk2xtmr.com     CN=www.orutxjqwf.net    CN=www.5orbut4ufhohm5rlj47.com  2013-06-19T00:00:00-0400        2014-04-20T00:00:00-0400
2013-12-30T15:20:21-0500        CwzpD92UikR0USUErj      10.0.0.126      58912   91.121.113.70   9001    TLSv10  TLS_DHE_RSA_WITH_AES_256_CBC_SHA        www.dv2nzruzkuf2ncqzpxh5vpg.com       CN=www.an2nldahkafrkz6qx.net    CN=www.ejybbncghc3qjraztwpr.com 2013-12-30T19:35:37-0500        2014-12-30T19:35:37-0500
2013-12-30T15:20:21-0500        CqAdrg1JryZY3kTrZ5      10.0.0.126      46649   5.135.187.167   9001    TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.3h2eyn3jwsjkggg3.com      CN=www.mt5unawhy.net    CN=www.nexscb2bdms.com  2013-12-16T00:00:00-0500        2014-01-10T23:59:59-0500
2013-12-30T15:20:21-0500        CWYgR82bEI9IjcHp7a      10.0.0.126      37960   212.83.158.5    443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.w5wtl.comCN=www.6suxdq3miwwewq4.net       CN=www.2fwld67ac2.com   2013-08-12T00:00:00-0400        2014-04-16T23:59:59-0400
2013-12-30T15:20:21-0500        CpGUEo3d5jBpzI6L04      10.0.0.126      50935   212.83.158.50   443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.lm6zdbm5w2jd5wxtmsfpkn.com        CN=www.ouzbot23a6lw3vvmszx.net  CN=www.s426lumoi7.com   2013-10-18T00:00:00-0400        2014-06-16T00:00:00-0400
2013-12-30T15:20:21-0500        CYocU22O3RREM4dfnl      10.0.0.126      49609   88.159.20.120   443     TLSv10  TLS_DHE_RSA_WITH_AES_256_CBC_SHA        www.exr2poqlv774jn4ddyvf5vvv.com      CN=www.qzsg2ioaoplbs2gaha5.net  CN=www.s5rc22gpzrwt4e.com       2013-12-30T19:54:02-0500        2014-12-30T19:54:02-0500
2013-12-30T15:20:21-0500        CxG1gw2N7G5uvDpiD2      10.0.0.126      57656   95.211.225.167  443     TLSv10  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      www.mwqdszwnojnepwmw4souyw.com        CN=www.rnspic4yus5crf6w.net     CN=www.75ba5lymxpbhw3a2kb.com   2013-11-19T00:00:00-0500        2014-02-05T23:59:59-0500
2013-12-30T15:20:21-0500        CcVZHF3a5TkT9byG2e      10.0.0.126      60680   80.100.45.156   443     TLSv10  TLS_DHE_RSA_WITH_AES_128_CBC_SHA        www.emqfcc55o7a4u4ecq3w63.com CN=www.c7kriuquvh.net   CN=www.npmxal2ohuefme26yf.com   2013-12-18T00:00:00-0500        2014-02-14T23:59:59-0500

Pretty straight forward process to identify Tor usage on a network. This could be coupled with matching the destination addresses with the Tor server list available here to provide further validation of Tor traffic.

Posted in network, security | Tagged , , | Leave a comment

Resizing Xen guest parition based filesystems

This post assumes you are running the Xen hypervisor and are using a partitions based filesystems for you Xen guest you would like to re-size. I have previously written on Installing Xen on CentOS 6 from source and another blog entry that describes how to create partition based Xen guests on Creating Debian guests on Xen using partition based filesystem if you would like to see how to get started running Xen.

To resize, first shutdown the guest instance:

$ sudo xm shutdown Wheezy
$ sudo lvresize /dev/VolGroup00/Wheezy -L +10GB
Extending logical volume Wheezy to 20.00 GiB
Logical volume Wheezy successfully resized
$ sudo lvdisplay
--- Logical volume ---
LV Path                /dev/VolGroup00/Wheezy
LV Name                Wheezy
VG Name                VolGroup00
LV UUID                jQqEFZ-sd39-siY6-kqCZ-l8Lq-UWWk-3f4oh5
LV Write Access        read/write
LV Creation host, time host.localdomain, 2013-05-14 12:32:00 -0400
LV Status              available
# open                 0
LV Size                20.00 GiB
Current LE             5120
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:0

I would first backup the partition that is going to be modified. This is going to sound weird; but this process uses fdisk to delete and recreate the partition.

List you partition:

$ sudo fdisk -l /dev/VolGroup00/Wheezy

Disk /dev/VolGroup00/Wheezy: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00081c29

Device Boot      Start         End      Blocks   Id  System
/dev/VolGroup00/Wheezy1               1          63      498688   82  Linux swap / Solaris
Partition 1 does not end on cylinder boundary.
/dev/VolGroup00/Wheezy2              63        1306     9985024   83  Linux

When trying to directly re-size, an error occurs.

$ sudo resize2fs /dev/VolGroup00/Wheezy
resize2fs 1.41.12 (17-May-2010)
resize2fs: Bad magic number in super-block while trying to open /dev/VolGroup00/Wheezy
Couldn't find valid filesystem superblock.

We are now going to delete the partition, as warned before, make sure you have backups.

$ sudo fdisk /dev/VolGroup00/Wheezy

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').

Command (m for help): p

Disk /dev/VolGroup00/Wheezy: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00081c29

Device Boot      Start         End      Blocks   Id  System
/dev/VolGroup00/Wheezy1               1          63      498688   82  Linux swap / Solaris
Partition 1 does not end on cylinder boundary.
/dev/VolGroup00/Wheezy2              63        1306     9985024   83  Linux

Command (m for help): d
Partition number (1-4): 2

Command (m for help): p

Disk /dev/VolGroup00/Wheezy: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00081c29

Device Boot      Start         End      Blocks   Id  System
/dev/VolGroup00/Wheezy1               1          63      498688   82  Linux swap / Solaris
Partition 1 does not end on cylinder boundary.

Recreate the partition with the new size.

Command (m for help): n
Command action
e   extended
p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (63-2610, default 63):
Using default value 63
Last cylinder, +cylinders or +size{K,M,G} (63-2610, default 2610):
Using default value 2610

Command (m for help): p

Disk /dev/VolGroup00/Wheezy: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00081c29

Device Boot      Start         End      Blocks   Id  System
/dev/VolGroup00/Wheezy1               1          63      498688   82  Linux swap / Solaris
Partition 1 does not end on cylinder boundary.
/dev/VolGroup00/Wheezy2              63        2610    20465113   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

The follow command splits the partitions apart as using the simple Debian partitioning scheme may combine them.

$ sudo kpartx -a /dev/VolGroup00/Wheezy
$ cd /dev/mapper/
$ ls
control  VolGroup00-Wheezy  VolGroup00-Wheezy1  VolGroup00-Wheezy2

Next, check the filesystem for errors.

$ sudo e2fsck -f VolGroup00-Wheezy2
e2fsck 1.41.12 (17-May-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
VolGroup00-Wheezy2: 29159/624624 files (0.2% non-contiguous), 224352/2496256 blocks

We can now re-size the filesystem.

$ sudo resize2fs VolGroup00-Wheezy2
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on VolGroup00-Wheezy2 to 5116278 (4k) blocks.
The filesystem on VolGroup00-Wheezy2 is now 5116278 blocks long.

Reattach the filesystems that were previously split.

$ sudo kpartx -d /dev/VolGroup00/Wheezy
$ ls
control  VolGroup00-Wheezy

A quick look at the logical volume and we can see we grew from 10 to 20 Gigabytes.

$ sudo lvscan
ACTIVE            '/dev/VolGroup00/Wheezy' [20.00 GiB] inherit

You should now be able to boot the guest using the larger file system.

To delete the guest filesystem:

sudo vgremove lvmxen
sudo pvremove /dev/sdb1
sudo parted /dev/sdb
(parted) rm 1
(parted) quit
Posted in systems administration | Tagged , | 2 Comments

Creating Debian guests on Xen using parition based filesystem

This guide describes how to create a filesystem and guest for the Xen hypervisor. This assumes you have a working Xen install with Dom U. I have described setting up a Xen hypervisor from source in another posted titled Installing Xen on CentOS 6 from source.

Create a partition to store virtual machines on. We want to use a partition based verse file based file-system for our guests as the performance is much better.

$ sudo parted /dev/sdb
mklabel gpt
(parted) unit GB
(parted) mkpart VolGroup00 0GB 400GB
(parted) set 1 lvm on
(parted) quit
(parted) p
Model: DELL PERC 6/i (scsi)
Disk /dev/sdb: 3999GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End    Size   File system  Name        Flags
1      1049kB  400GB  400GB               VolGroup00  lvm

Create a partition for the first virtual machine.

$ sudo pvcreate /dev/sdb1
$ sudo vgcreate VolGroup00 /dev/sdb1
$ sudo vgdisplay
--- Volume group ---
VG Name               VolGroup00
System ID
Format                lvm2
Metadata Areas        1
Metadata Sequence No  1
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                0
Open LV               0
Max PV                0
Cur PV                1
Act PV                1
VG Size               372.53 GiB
PE Size               4.00 MiB
Total PE              95367
Alloc PE / Size       0 / 0
Free  PE / Size       95367 / 372.53 GiB
VG UUID               hdCkfh-twnj-Nu2V-FsTe-RsQg-PzlE-5w4QGR

Create a logical volume for the virtual machine.

$ sudo lvcreate -L 10GB -n Wheezy VolGroup00
$ sudo lvdisplay
--- Logical volume ---
LV Path                /dev/VolGroup00/Wheezy
LV Name                Wheezy
VG Name                VolGroup00
LV UUID                jQqEFZ-sd39-siY6-kqCZ-l8Lq-UWWk-3f4oh5
LV Write Access        read/write
LV Creation host, time host.localdomain, 2013-05-14 12:32:00 -0400
LV Status              available
# open                 0
LV Size                10.00 GiB
Current LE             2560
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:0

Get the latest Debian hd-media. Specify these parameters in the virtual machine configuration that will be used for the first start-up, i.e. the install of your guest. A second configuration will be used for booting the guest post-install.

kernel = "/scratch/debian/wheezy/vmlinuz"
ramdisk = "/scratch/debian/wheezy/initrd.gz"
extra = "debian-installer/exit/always_halt=true -- console=hvc0"
memory = 512
name = "Wheezy"
vif = ['bridge=br0']
disk = ['phy:/dev/VolGroup00/Wheezy,xvda,w']

Connect to the new guest with a console and perform the installation.

$ sudo xl create -c /etc/xen/install-debian.cfg

Start a guest without a console.

$ sudo xl create /etc/xen/debian.cfg

Leave the console.

$ "Ctrl+]"

List the instances.


$ sudo xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  2048     1     r-----    237.4
Wheezy                                      11   512     1     -b----      6.8

Connect to the console.

$ sudo xl console Wheezy

Leave the console.

$ "Ctrl+]"

If you have any questions or feel something is missing, leave a comment below.

Posted in systems administration | Tagged , | Leave a comment

Installing Xen on CentOS 6 from source

I recently had a need to install Xen hypervisor on CentOS and most of the guides covered using the package maintainers version. Further, RHEL distributions favor using KVM. I did come across HowTo: Install XEN Dom0 on CentOS 6 from source but the domain was blocked (Google cache made quick work of getting around that issue) and there were a few steps that felt unclear. I took that guide and made a few changes which are reflected below. You may want to also reference the Xen Wiki CenOS 6.2, Xen 4.2.1, and Kernel version 3.9.2 were used in this example but newer and older versions should be similar.

First install dependencies:

yum groupinstall "Development Libraries"
yum groupinstall "Development Tools"
yum install transfig wget tar less texi2html libaio-devel dev86 glibc-devel e2fsprogs-devel gitk mkinitrd iasl xz-devel bzip2-devel
yum install pciutils-libs pciutils-devel SDL-devel libX11-devel gtk2-devel bridge-utils PyXML qemu-common qemu-img mercurial texinfo
yum install libidn-devel yajl yajl-devel ocaml ocaml-findlib ocaml-findlib-devel python-devel uuid-devel libuuid-devel openssl-devel
yum install glibc-devel.i686
yum install
libvirt python-virtinst

Download the latest Xen source package.

$ tar xzf xen-4.2.1.tar.gz
$ cd xen-4.2.1
$ ./configure
$ make xen && make tools && make stubdom
$ sudo make install xen
$ sudo make install xen-tools
$ sudo make install stubdom

Prevent the screen from powering off:

$ sudo sh -c "echo '/usr/bin/setterm -powersave off' >> /etc/rc.local"

Define the resources for domain 0:

$ sudo sh -c "echo 'xl sched-credit -d Domain-0 -w 512' >> /etc/xendom0caps"
$ sudo chmod +x /etc/xendom0caps

Start the services at boot:

sudo ln -s /etc/init.d/xendomains /etc/rc0.d/S10xendomains
sudo ln -s /etc/init.d/xendomains /etc/rc6.d/S10xendomains
sudo ln -s /etc/init.d/xendomains /etc/rc3.d/S98xendomains
sudo ln -s /etc/init.d/xencommons /etc/rc3.d/S98xencommons
sudo ln -s /etc/xendom0caps /etc/rc3.d/S97xendom0caps

Optionally for those that want to use the xm commands.

sudo ln -s /etc/init.d/xend /etc/rc3.d/S98xend

Make sure everything is going to start at the correct runlevel. Note that xend is optional

$ chkconfig --list |grep xen
xencommons      0:off   1:off   2:off   3:on    4:off   5:off   6:off
xend            0:off   1:off   2:off   3:on    4:off   5:off   6:off
xendomains      0:on    1:off   2:off   3:on    4:off   5:off   6:on

Make sure the weight is setup, this may vary depending your needs/resources available.

$ sudo xl sched-credit
Cpupool Pool-0: tslice=30ms ratelimit=1000us
Name                                ID Weight  Cap
Domain-0                             0    512    0
Wheezy                               3    256    0

Download the latest kernel version you would like to use and extract the contents of the archive. You can try pulling your configuration via “make oldconfig“, so your old settings are migrated and only new or changed options are presented to you to select. Then to make sure everything is ok, run “make menuconfig” or “make xconfig” to determine if the feature/module setting are appropriate for you. I left everything alone with the exception of enabling the Xen features as described below. make oldconfig is clever, it can do it’s job between different versions of kernel although just issuing a “make menuconfig” is probably also fine.

$ cd linux-3.9.2
$ make oldconfig
scripts/kconfig/conf --oldconfig Kconfig
#
# configuration written to .config
#

Alternatively just use the defaults and add the required Xen features:

$ cd linux-3.9.2
$ make menuconfig

Location:
-> Processor type and features
-> Paravirtualized guest support
Select all features.

makemenu3

makemenu2

makemenu4

Location:
-> Device Drivers
-> Block devices
Select the two features “Xen virtual block device support” and “Xen block-device backend driver”

makemenu5

makemenu6

makemenu7

Location:
-> Device Drivers
-> Xen driver support
Select all features.

makemenu5

makemenu8

makemenu9

Location:
-> Device Drivers
-> Network device support
Select the two features “Xen network device frontend driver” and “Xen backend network device”

makemenu5

makemenu10

makemenu11

Lastly, you can search using “/” when at the root menu to see what you have enabled:

makemenu1

Which will provide you a list of features that have been selected but it may be easier to grep through the .config as shown in the next command.

makemenu12

You can use “grep” to ensure you should have similar values for your Xen settings after running menu config.

$ grep XEN .config
CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=500
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_XEN_DEBUG_FS=y
CONFIG_PCI_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_NETXEN_NIC=m
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
# CONFIG_XEN_WDT is not set
CONFIG_XEN_FBDEV_FRONTEND=y
CONFIG_XEN_BALLOON=y
CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=y
CONFIG_XEN_GRANT_DEV_ALLOC=y
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_PCIDEV_BACKEND=y
CONFIG_XEN_PRIVCMD=y
CONFIG_XEN_ACPI_PROCESSOR=y
CONFIG_XEN_MCE_LOG=y
CONFIG_XEN_HAVE_PVMMU=y

If all of the Xen features are enabled, move on to compiling.

$ make bzImage
$ make modules
$ sudo make modules_install

Copy the images to the appropriate locations.

$ sudo cp -a arch/x86/boot/bzImage /boot/vmlinuz-3.9.2
$ sudo cp -a System.map /boot/System.map-3.9.2
$ sudo cp -a .config /boot/config-3.9.2
$ sudo depmod -a
$ sudo mkinitrd /boot/initrd.img-3.9.2 3.9.2

Add a grub entry to /etc/grub.conf, make sure it is the first entry but leave an existing distribution kernel entry to fall back to if there are problems:

title Xen 4.2.1 / Kernel 3.9.2
root (hd0,0)
kernel /xen.gz
module /vmlinuz-3.9.2
module /initrd.img-3.9.2

Reboot the system and you should be able to run the following command to verify that your efforts have paid off.

$ sudo xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  2048     1     r-----     941.4

Now you can move on to setting up a guest as described in Creating Debian guests on Xen using parition based filesystem.

If you are unable to reboot using your new kernel, revert back to a distro kernel and double check that you have done everything as described. If something is not clear or could be improved upon, let me know by leaving a comment below.

Posted in systems administration | Tagged , | Leave a comment

Passive DNS collection and analysis using YaF and Mediator

Passive DNS is a useful tool for any analysts teams toolbox, I have noted several public sensors here but they only see data (queries and responses) that transverse their sensors. I have been working on setting up passive DNS using Yet another Flowmeter (YaF) and Mediator (YaF to MySQL) to fill the gap where third-party sensors may not be providing the coverage I would like. Passive DNS can provide tremendous insight and analytics upon DNS queries that users and/or malware may be performing. A few items of interest:

  • Hostnames that have a large number of IP addresses associated with them in a short time period and they have only been visited by very few hosts host on the network.
  • Tertiary name usage associated with a specific domain?
  • When was the domain first resolved on the network and further, how often is it being resolved and by whom?
  • A recently accessed/registered domain with short time to live (TTL’s) often associated with new IP addresses may indicate malicious activity, or a CDN.
  • Queries for TLD’s that you typically do not interact with may be worth looking into.
  • Users using non-approved DNS servers

Passive DNS may be also helpful in tracking infections using Fast-flux which make blocking the C2 difficult as the attackers will create algorithms to rotate the IP addresses and even the hostnames in the case of double-flux. (TorPig) The list goes on but in a nutshell, I wanted to be able to perform this activity without having to rely on having all of the DNS server logs in a centralized location, especially since users may reconfigure their DNS settings to use non-approved servers, e.g. BYOD.

This entry demonstrates how to build and setup YaF and Mediator both of which are available from the CERT NetSA site and should be considered complementary to the documentation the NetSA team have already provided for each of the respective tools. This setup was tested on CentOS 6.4 but most Linux distributions should work fine.

1. Have site reconfigure interfaces on all hosts. eth0 should be management interface and eth1 should be the tap OR whatever makes sense, this need to happen every time the host comes up, i.e.

sudo ifconfig eth1 up promisc

2. Ensure development libraries/dependencies are installed. Some may require enabling the optional software channel

sudo yum install glib2-devel lzo  gcc-c++ libpcap-devel pcre-devel

3. Install libfixbuf

cd libfixbuf-1.3.0
./configure
make
sudo make install

4. Install YaF

cd yaf-2.3.3
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/
./configure --with-libpcap --enable-applabel --enable-plugins
make
sudo make install

5. Edit ld

sudo echo "/usr/local/lib" >> /etc/ld.so.conf
sudo /sbin/ldconfig
sudo /sbin/ldconfig -v | grep libzmq # should rebuild the cache including zmq too.

OR

export PATH=$PATH:/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

6. Configure cmake

cd cmake-2.8.10.2
./configure
gmake

6. Optionally, configure YaF to File output for testing purposes.

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/
cd yaf_file_mediator-1.1.0/
./configure
../cmake-2.8.10.2/bin/cmake .
make

7. Configure YaF to MySQL

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/
cd yaf_silk_mysql_mediator-1.4.0
../cmake-2.8.10.2/bin/cmake .
./configure --with-mysql
make

Next, populate create a database and respective tables:

./yafMySQL -o localhost -n username -p password -d eflows

8. Setup YaF to start capturing. Here we are only capture DNS traffic and rotating the files written to disk after 5 minutes. Originally set to 10 minutes but yaf_silk_mysql_mediator would segmentation fault because MySQL would close the connection before all of the data would insert. We have a continuous method that works a little better which we should a little later. We lock the file so that another process cannot take the file that is currently being written to.

sudo /usr/local/bin/yaf --live pcap --in eth1 --out /data/ipfix/ --rotate 600 --filter="port 53" --applabel --applabel-rules=/usr/local/etc/yafApplabelRules.conf --max-payload=1000 --plugin-name=/usr/local/lib/yaf/dpacketplugin.la --plugin-opts="53" --lock --become-user=nobody --become-group=nobody &

9. Testing the output of a YaF

yaf_file_mediator-1.1.0/yaf_file_mediator --input /data/ipfix/filename.yaf --output test.txt

After a few minutes, you should be able to parse the filename.yaf that was first written (in this case 5 minutes). The contents of test.txt should be similar to the following:

-------------------------------
Template ID is 45841
Application Label: 53
Source IP: 192.168.0.5
Destination IP: 8.8.8.8
Source Port: 53855
Dest Port: 53
Flow Attributes: 1
Rev Flow Attributes: 0
flowStartTime: 2013-04-24 23:53:43
flowEndTime: 2013-04-24 23:58:02
flowEndReason: 1
Protocol: 17
Octet Total Count: 120
Rev Octet count: 244
Packet Total Count: 2
Rev Packet Total Count: 2
DNS ID: 32852 Type: 28 RR Section: 0 TTL: 0 Query: www.google.com.
DNS ID: 32852 Type: 28 RR Section: 1 TTL: 204 RRName: www.google.com. AAAA: 2607:f8b0:400c:0c04::0069

-------------------------------
Template ID is 45841
Application Label: 53
Source IP: 192.168.0.5
Destination IP: 8.8.8.8
Source Port: 50845
Dest Port: 53
flowStartTime: 2013-04-24 23:58:02
flowEndTime: 2013-04-24 23:58:02
flowEndReason: 1
Protocol: 17
Octet Total Count: 60
Rev Octet count: 156
Packet Total Count: 1
Rev Packet Total Count: 1
DNS ID: 21141 Type: 1 RR Section: 0 TTL: 0 Query: www.google.com.
DNS ID: 21141 Type: 1 RR Section: 1 TTL: 208 RRName: www.google.com. A: 74.125.26.103
DNS ID: 21141 Type: 1 RR Section: 1 TTL: 208 RRName: www.google.com. A: 74.125.26.99
DNS ID: 21141 Type: 1 RR Section: 1 TTL: 208 RRName: www.google.com. A: 74.125.26.105
DNS ID: 21141 Type: 1 RR Section: 1 TTL: 208 RRName: www.google.com. A: 74.125.26.104
DNS ID: 21141 Type: 1 RR Section: 1 TTL: 208 RRName: www.google.com. A: 74.125.26.106
DNS ID: 21141 Type: 1 RR Section: 1 TTL: 208 RRName: www.google.com. A: 74.125.26.147

10. After you have confirmed that your YaF entries contain records, add a little automation. This will scoop up the files in the directory where the YaF files are being written, place them in the MySQL DBMS and delete the file. Note, if you start seeing “Segmentation Fault” then MySQL may be closing the connection before all of the records from the YaF file could be written to the DBMS. You can try modifying MySQL parameters or reduce the the size of YaF files being written to disk in order to try mitigating this symptom if it occurs in your environment.

for i in $( ls /data/ipfix/*.yaf ); do /home/user/silk-installs/yaf_silk_mysql_mediator-1.4.0/yaf_silk_mysql_mediator --in-file $i --mysql-host localhost --name username --pass password --database eflows && sudo rm $i; done

Here is our first query, lets see who has recently made requests for www.google.com.

mysql> SELECT rrname,rrval,srcip4,dstip4,flowStartMilliseconds FROM dns d, flows f WHERE f.id = d.id AND rrname LIKE "www.google.com." GROUP by rrval ORDER BY f.id DESC LIMIT 50;
+-----------------+---------------------------+------------+-----------+-----------------------+
| rrname          | rrval                     | srcip4     | dstip4    | flowStartMilliseconds |
+-----------------+---------------------------+------------+-----------+-----------------------+
| www.google.com. | 2001:4860:4001:0802::1012 | 3232235525 | 134744072 | 2013-05-03 17:47:24   |
| www.google.com. | 2001:4860:4001:0801::1014 | 3232235525 | 134744072 | 2013-05-03 15:35:32   |
| www.google.com. | 2001:4860:4001:0802::1014 | 3232235525 | 134744072 | 2013-05-03 11:28:42   |
| www.google.com. | 2001:4860:4001:0801::1010 | 3232235525 | 134744072 | 2013-05-02 16:48:31   |
| www.google.com. | 2001:4860:4001:0802::1011 | 3232235525 | 134744072 | 2013-05-02 13:33:57   |
| www.google.com. | 2001:4860:4001:0803::1010 | 3232235525 | 134744072 | 2013-05-02 12:01:56   |
| www.google.com. | 2607:f8b0:4004:0801::1012 | 3232235525 | 134744072 | 2013-05-01 21:36:55   |
| www.google.com. | 2001:4860:4001:0802::1010 | 3232235525 | 134744072 | 2013-05-01 12:44:52   |
| www.google.com. | 74.125.239.80             | 3232235525 | 134744072 | 2013-05-01 10:45:04   |
| www.google.com. | 74.125.239.83             | 3232235525 | 134744072 | 2013-05-01 10:45:04   |
| www.google.com. | 74.125.239.82             | 3232235525 | 134744072 | 2013-05-01 10:45:04   |
| www.google.com. | 74.125.239.81             | 3232235525 | 134744072 | 2013-05-01 10:45:04   |
| www.google.com. | 74.125.239.84             | 3232235525 | 134744072 | 2013-05-01 10:45:04   |
| www.google.com. | 2607:f8b0:4004:0802::1010 | 3232235525 | 134744072 | 2013-04-29 19:54:00   |
| www.google.com. | 2607:f8b0:4005:0802::1010 | 3232235525 | 134744072 | 2013-04-28 15:52:00   |
| www.google.com. | 2607:f8b0:4004:0803::1013 | 3232235525 | 134744072 | 2013-04-28 15:05:53   |
| www.google.com. | 2607:f8b0:4005:0802::1011 | 3232235525 | 134744072 | 2013-04-27 14:45:35   |
| www.google.com. | 2607:f8b0:4004:0801::1013 | 3232235525 | 134744072 | 2013-04-26 18:53:45   |
| www.google.com. | 2607:f8b0:4005:0802::1012 | 3232235525 | 134744072 | 2013-04-26 13:55:51   |
| www.google.com. | 2607:f8b0:4005:0802::1013 | 3232235525 | 134744072 | 2013-04-26 12:35:18   |
| www.google.com. | 74.125.239.145            | 3232235525 | 134744072 | 2013-04-26 12:03:10   |
| www.google.com. | 74.125.239.148            | 3232235525 | 134744072 | 2013-04-26 12:03:10   |
| www.google.com. | 74.125.239.146            | 3232235525 | 134744072 | 2013-04-26 12:03:10   |
| www.google.com. | 74.125.239.147            | 3232235525 | 134744072 | 2013-04-26 12:03:10   |
| www.google.com. | 74.125.239.144            | 3232235525 | 134744072 | 2013-04-26 12:03:10   |
| www.google.com. | 2607:f8b0:4005:0802::1014 | 3232235525 | 134744072 | 2013-04-26 11:31:59   |
| www.google.com. | 74.125.228.112            | 3232235525 | 134744072 | 2013-04-25 16:25:39   |
| www.google.com. | 74.125.228.114            | 3232235525 | 134744072 | 2013-04-25 16:25:39   |
| www.google.com. | 74.125.228.113            | 3232235525 | 134744072 | 2013-04-25 16:25:39   |
| www.google.com. | 74.125.228.115            | 3232235525 | 134744072 | 2013-04-25 16:25:39   |
| www.google.com. | 74.125.228.116            | 3232235525 | 134744072 | 2013-04-25 16:25:39   |
| www.google.com. | 2607:f8b0:4004:0802::1012 | 3232235525 | 134744072 | 2013-04-25 11:29:45   |
| www.google.com. | 2607:f8b0:4004:0803::1014 | 3232235525 | 134744072 | 2013-04-24 20:33:42   |
| www.google.com. | 2607:f8b0:400e:0c04::006a | 3232235525 | 134744072 | 2013-04-24 18:04:19   |
| www.google.com. | 2607:f8b0:400e:0c02::006a | 3232235525 | 134744072 | 2013-04-24 15:26:22   |
| www.google.com. | 74.125.228.20             | 3232235525 | 134744072 | 2013-04-24 12:05:43   |
| www.google.com. | 74.125.228.16             | 3232235525 | 134744072 | 2013-04-24 12:05:43   |
| www.google.com. | 74.125.228.18             | 3232235525 | 134744072 | 2013-04-24 12:05:43   |
| www.google.com. | 74.125.228.19             | 3232235525 | 134744072 | 2013-04-24 12:05:43   |
| www.google.com. | 74.125.228.17             | 3232235525 | 134744072 | 2013-04-24 12:05:43   |
| www.google.com. | 2607:f8b0:4004:0801::1014 | 3232235525 | 134744072 | 2013-04-23 20:43:26   |
| www.google.com. | 74.125.228.50             | 3232235525 | 134744072 | 2013-04-23 20:38:43   |
| www.google.com. | 74.125.228.51             | 3232235525 | 134744072 | 2013-04-23 20:38:43   |
| www.google.com. | 74.125.228.52             | 3232235525 | 134744072 | 2013-04-23 20:38:43   |
| www.google.com. | 74.125.228.48             | 3232235525 | 134744072 | 2013-04-23 20:38:43   |
| www.google.com. | 74.125.228.49             | 3232235525 | 134744072 | 2013-04-23 20:38:43   |
| www.google.com. | 2607:f8b0:4004:0801::1011 | 3232235525 | 134744072 | 2013-04-23 18:38:52   |
| www.google.com. | 2607:f8b0:400e:0c01::0067 | 3232235525 | 134744072 | 2013-04-23 15:57:45   |
| www.google.com. | 2607:f8b0:4004:0801::1010 | 3232235525 | 134744072 | 2013-04-23 15:07:59   |
| www.google.com. | 2607:f8b0:400e:0c01::0069 | 3232235525 | 134744072 | 2013-04-23 12:30:28   |
+-----------------+---------------------------+------------+-----------+-----------------------+

Here is a similar query but we want to see any tertiary youtube.com domains and sort by the lookup returned.

mysql> SELECT qr,type,auth,nx,ttl,rrname,rrval from dns WHERE rrname LIKE "%.youtube.com." GROUP BY rrval LIMIT 50;
+------+------+------+------+------+--------------------------------+----------------+
| qr   | type | auth | nx   | ttl  | rrname                         | rrval          |
+------+------+------+------+------+--------------------------------+----------------+
|    0 |    1 |    0 |    0 |    0 | www.youtube.com.               |                |
|    1 |    1 |    0 |    0 |  300 | v17.lscache2.c.youtube.com.    | 12.216.80.12   |
|    1 |    1 |    0 |    0 | 1800 | r2.sn-5uu-vgqe.c.youtube.com.  | 12.216.80.13   |
|    1 |    1 |    0 |    0 | 1800 | r3.sn-5uu-vgqe.c.youtube.com.  | 12.216.80.14   |
|    1 |    1 |    0 |    0 | 1800 | r4.att-ord1.c.youtube.com.     | 12.216.80.15   |
|    1 |    1 |    0 |    0 | 1800 | r6.sn-5uu-vgqe.c.youtube.com.  | 12.216.80.17   |
|    1 |    1 |    0 |    0 | 1714 | r8.sn-5uu-vgqe.c.youtube.com.  | 12.216.80.19   |
|    1 |    1 |    0 |    0 | 1741 | r1.sn-5uu-vgql.c.youtube.com.  | 12.216.80.44   |
|    1 |    1 |    0 |    0 | 1800 | r2.sn-5uu-vgql.c.youtube.com.  | 12.216.80.45   |
|    1 |    1 |    0 |    0 | 1800 | r3.sn-5uu-vgql.c.youtube.com.  | 12.216.80.46   |
|    1 |    1 |    0 |    0 | 1279 | r4.sn-5uu-vgql.c.youtube.com.  | 12.216.80.47   |
|    1 |    1 |    0 |    0 | 1800 | r6.sn-5uu-vgql.c.youtube.com.  | 12.216.80.49   |
|    1 |    1 |    0 |    0 | 1800 | r7.sn-5uu-vgql.c.youtube.com.  | 12.216.80.50   |
|    1 |    1 |    0 |    0 | 1739 | r8.sn-5uu-vgql.c.youtube.com.  | 12.216.80.51   |
|    1 |    1 |    0 |    0 | 1800 | r12.sn-hp576nes.c.youtube.com. | 173.194.17.17  |
|    1 |    1 |    0 |    0 | 1800 | r20.sn-hp576nes.c.youtube.com. | 173.194.17.25  |
|    1 |    1 |    0 |    0 | 1800 | r6.sn-q4f7dnel.c.youtube.com.  | 173.194.24.11  |
|    1 |    1 |    0 |    0 | 1800 | r1.dfw06s08.c.youtube.com.     | 173.194.24.134 |
|    1 |    1 |    0 |    0 | 1800 | r15.sn-q4f7dn7r.c.youtube.com. | 173.194.24.148 |
|    1 |    1 |    0 |    0 | 1800 | r18.sn-hp576n7d.c.youtube.com. | 173.194.29.119 |
|    1 |    1 |    0 |    0 | 1800 | r9.sn-hp576n7z.c.youtube.com.  | 173.194.29.46  |
|    1 |    1 |    0 |    0 | 1800 | r5.sn-ab5e6ner.c.youtube.com.  | 173.194.31.10  |
|    1 |    1 |    0 |    0 | 1800 | r1.sn-ab5e6nle.c.youtube.com.  | 173.194.31.102 |
|    1 |    1 |    0 |    0 |  640 | r2.sn-ab5e6nle.c.youtube.com.  | 173.194.31.103 |
|    1 |    1 |    0 |    0 | 1800 | r3.sn-ab5e6nle.c.youtube.com.  | 173.194.31.104 |
|    1 |    1 |    0 |    0 | 1800 | r4.sn-ab5e6nle.c.youtube.com.  | 173.194.31.105 |
|    1 |    1 |    0 |    0 | 1800 | r5.sn-ab5e6nle.c.youtube.com.  | 173.194.31.106 |
|    1 |    1 |    0 |    0 | 1800 | r6.sn-ab5e6nle.c.youtube.com.  | 173.194.31.107 |
|    1 |    1 |    0 |    0 | 1800 | r7.sn-ab5e6nle.c.youtube.com.  | 173.194.31.108 |
|    1 |    1 |    0 |    0 | 1800 | r8.sn-ab5e6nle.c.youtube.com.  | 173.194.31.109 |
|    1 |    1 |    0 |    0 |  705 | r6.sn-ab5e6ner.c.youtube.com.  | 173.194.31.11  |
|    1 |    1 |    0 |    0 | 1800 | r9.sn-ab5e6nle.c.youtube.com.  | 173.194.31.110 |
|    1 |    1 |    0 |    0 | 1800 | r10.sn-ab5e6nle.c.youtube.com. | 173.194.31.111 |
|    1 |    1 |    0 |    0 |  292 | r11.sn-ab5e6nle.c.youtube.com. | 173.194.31.112 |
|    1 |    1 |    0 |    0 | 1800 | r12.sn-ab5e6nle.c.youtube.com. | 173.194.31.113 |
|    1 |    1 |    0 |    0 |  178 | r13.sn-ab5e6nle.c.youtube.com. | 173.194.31.114 |
|    1 |    1 |    0 |    0 | 1800 | r14.sn-ab5e6nle.c.youtube.com. | 173.194.31.115 |
|    1 |    1 |    0 |    0 | 1800 | r15.sn-ab5e6nle.c.youtube.com. | 173.194.31.116 |
|    1 |    1 |    0 |    0 | 1800 | r16.sn-ab5e6nle.c.youtube.com. | 173.194.31.117 |
|    1 |    1 |    0 |    0 | 1800 | r17.sn-ab5e6nle.c.youtube.com. | 173.194.31.118 |
|    1 |    1 |    0 |    0 | 1800 | r18.sn-ab5e6nle.c.youtube.com. | 173.194.31.119 |
|    1 |    1 |    0 |    0 | 1653 | r7.sn-ab5e6ner.c.youtube.com.  | 173.194.31.12  |
|    1 |    1 |    0 |    0 | 1800 | r19.sn-ab5e6nle.c.youtube.com. | 173.194.31.120 |
|    1 |    1 |    0 |    0 | 1800 | r20.sn-ab5e6nle.c.youtube.com. | 173.194.31.121 |
|    1 |    1 |    0 |    0 | 1800 | r8.sn-ab5e6ner.c.youtube.com.  | 173.194.31.13  |
|    1 |    1 |    0 |    0 |   81 | r1.sn-ab5e6nll.c.youtube.com.  | 173.194.31.134 |
|    1 |    1 |    0 |    0 | 1800 | r2.sn-ab5e6nll.c.youtube.com.  | 173.194.31.135 |
|    1 |    1 |    0 |    0 | 1800 | r3.sn-ab5e6nll.c.youtube.com.  | 173.194.31.136 |
|    1 |    1 |    0 |    0 | 1800 | r4.sn-ab5e6nll.c.youtube.com.  | 173.194.31.137 |
|    1 |    1 |    0 |    0 | 1800 | r5.lga15s22.c.youtube.com.     | 173.194.31.138 |
+------+------+------+------+------+--------------------------------+----------------+
50 rows in set (22.17 sec)

An alternative method is to write YaF records directly to mediator, and further the MySQL DBMS rather then writing files to disk although you can still do this with the appropriate toggles. Here is example usage to start the processes:

$ ./silk-installs/yaf_silk_mysql_mediator-1.4.0/yaf_silk_mysql_mediator --in-host=127.0.0.1 --in-port=18000 --mysql-host=localhost --name=username --pass password --database eflows
$ sudo /usr/local/bin/yaf --live pcap --in eth1 --out 127.0.0.1 --ipfix-port=18000 --ipfix tcp --log=/var/log/yaf.log --filter="port 53" --applabel --applabel-rules=/usr/local/etc/yafApplabelRules.conf --max-payload=1000 --plugin-name=/usr/local/lib/yaf/dpacketplugin.la --plugin-opts="53" &

Ensure YaF and mediator are connected:

$ sudo netstat -tupan|grep yaf
tcp        0      0 127.0.0.1:18000             0.0.0.0:*                   LISTEN      6497/yaf_silk_mysql
tcp        0      0 127.0.0.1:47417             127.0.0.1:18000             ESTABLISHED 6513/yaf
tcp        0      0 127.0.0.1:18000             127.0.0.1:47417             ESTABLISHED 6497/yaf_silk_mysql

You may use the following MySQL query to see when the table was last updated to ensure records are being inserted on a regular basis:

mysql> SHOW TABLE STATUS in eflows;

After a few minutes of collection, query a domain that has been recently resolved and you should see it in the DBMS.

mysql> SELECT rrname,rrval from dns WHERE rrname LIKE "%rsreese.com." GROUP BY rrval LIMIT 10;
+--------------+--------------------------------+
| rrname       | rrval                          |
+--------------+--------------------------------+
| rsreese.com. |                                |
| rsreese.com. | 2600:3c02::f03c:91ff:fe96:f7bd |
| rsreese.com. | 74.207.234.79                  |
| rsreese.com. | ns1.linode.com.                |
| rsreese.com. | ns2.linode.com.                |
| rsreese.com. | ns3.linode.com.                |
| rsreese.com. | ns4.linode.com.                |
| rsreese.com. | ns5.linode.com.                |
+--------------+--------------------------------+
8 rows in set (18.26 sec)

There are a number of different fields available for query so I leave it to you to come up with whatever is most useful for you. Further, think of how you could write a shiny front-end for analysts to use rather then having to use the MySQL command line interface. Hope you found this useful and leave a comment if you did or have any questions.

Posted in security | Tagged , , | Leave a comment

Online Information Security Analysis Tools and Resources

Security_Panda

Panda courtesy of www.xen.org.

Here are a list of sites that analysts may find useful in their day-to-day analysis of indicators and threats. While verifying and searching for new sources, I came across Links and resources for malware samples and Free Online Tools for Looking Up Potentially Malicious Websites which may also be helpful. This page may be considered a work-in-progress but if you feel something is missing or broken, leave a comment or contact me. Entries with an asterisk (*) require an account.

IP/ISP/Domain, and WHOIS look-ups

IP and Domain analysis for malware or web-based threats

Open-source Threat Reports, IP and Domain Blacklists

Malware Binary Analysis

Malware Samples

HTTP Agent sniffers, Decode De-Obfuscate JavaScript and Base 64

BotNet Tracking

Site History

Google Hacking

Posted in security | Tagged , , | 4 Comments

Running Moloch

This is an overview of installing and running Moloch on a single host. After seeing the 2013 ShmooCon presentation, I have been looking forward to giving the tool a test-drive. Per the documentation, “Moloch is a open source large scale IPv4 full PCAP capturing, indexing and database system”. It is fast and has a pretty nice interface to boot. Although it does not contain the same feature-set as some commercial over the shelf (COTS) products, I see Moloch fitting into a similar space where COTS products such might sit. When analysts are made aware of anomaly-based alerts from signature/misuse based intrusion detection systems (IDS), e.g. Snort, or anomalous activity from network flow, e.g. SiLK, the analyst can obtain packet capture (PCAP) for further investigation. The existing commercial tool suites are expensive PCAP indexing tools if that is all they are being used for, especially if you are locked into their storage mechanism. A budget conscious security operation center (SOC) can setup Moloch for a fraction of the maintenance cost of commercial offerings and instead use the funds for additional hardware (longer retention), maintenance, and even some Moloch development contribution.

Although the developers have provided a script to get Moloch going, I had a few hiccups so I figured I would document them in the event they help someone else out. I used a CentOS release 6.4 (Final) x86_64 base bare-metal install. I imagine you could run it in a virtual environment for testing purposes. After you get the operating system (OS) installed and patched, pull down the latest Oracle Java for your distribution. Untar the package and create a symbolic in a directory that Moloch will be able to find.

$ sudo cp -R jre1.7.0_17/ /usr/bin/
$ sudo  ln -s /usr/bin/jre1.7.0_17/bin/java /usr/bin/java

Next, pull down the latest moloch build. I just grabbed the ZIP but it’s hosted on GitHub. You might want to take a look at the install script to see if everything is ideal for you. Run the easy installer which should pull down the prerequisites needed, build and install.

$ cd moloch-master/
$ sudo ./easybutton-singlehost.sh

If everything went smoothly, the script will try starting the three Moloch components being elasticsearch, capture, and viewer. The latter process did not start and this was probably for the better as I required me to take a closer look at what the install script was doing and the default configuration files (config.ini and elaseticsearch.yml). The configuration files are located in:

# ls -l /data/moloch/etc/
total 4680
-rw-r--r--. 1 root root    6766 Mar 14 17:21 config.ini
-rw-r--r--. 1 root root    6551 Mar 13 22:30 config.ini.template
-rw-r--r--. 1 root root   12545 Mar 14 22:54 elasticsearch.yml
-rw-r--r--. 1 root root 3360134 Mar  6 15:10 GeoIPASNum.dat
-rw-r--r--. 1 root root 1358092 Mar  5 21:48 GeoIP.dat
-rw-r--r--. 1 root root    1249 Mar 13 22:31 moloch.crt
-rw-r--r--. 1 root root    1029 Mar 13 22:31 moloch.csr
-rw-r--r--. 1 root root    1704 Mar 13 22:31 moloch.key
-rw-r--r--. 1 root root   10875 Mar 13 22:31 openssl.cnf
-rw-r--r--. 1 root root   10909 Mar 13 22:30 openssl.cnf.template

First, I had to sort out what was preventing the viewer from starting so I took a look at the viewer.log.

Mar 13 23:13:04 http.c:245 moloch_http_connect(): Connecting 0x7f6e0d19b010
Mar 13 23:13:04 http.c:276 moloch_http_connect(): 0x7f6e0d19b010: Error: Error connecting: Address family not supported by protocol
Couldn't connect to elastic search at 'localhost:9200'

Log files are located in:

# ls -l /data/moloch/logs/
total 6047776
-rw-r--r--. 1 root root 6180585472 Mar 15 23:44 capture.log
-rw-r--r--. 1 root root   12062720 Mar 14 17:22 capture.log.old
-rw-r--r--. 1 root root          0 Mar 13 22:31 Moloch_index_indexing_slowlog.log
-rw-r--r--. 1 root root          0 Mar 13 22:31 Moloch_index_search_slowlog.log
-rw-r--r--. 1 root root        163 Mar 15 20:00 Moloch.log
-rw-r--r--. 1 root root       2943 Mar 13 23:27 Moloch.log.2013-03-13
-rw-r--r--. 1 root root      35410 Mar 14 23:34 Moloch.log.2013-03-14
-rw-r--r--. 1 root root     208487 Mar 15 23:06 viewer.log
-rw-r--r--. 1 root root       1668 Mar 15 09:06 viewer.log.old

I had to change the directive in the config.ini from localhost to 127.0.0.1, otherwise the viewer would not connect to the elasticsearch instance in CentOS. Probably due to the initial IPv6 look-up, just a guess. Also added a Berkley packet filter (BPF) to prevent the capture and indexing of internal-to-internal traffic.

elasticsearch=127.0.0.1:9200
bpf=not src net (10.0.0.0/8) and dst net (10.0.0.0/8)

While I was adjusting the configuration, I decided to adjust the elasticsearch memory usage from what I originally specified in the installer script. You might want to take a look at their hardware requirements but I was able to run with a less powerful node:

$ sudo vim /data/moloch/bin/run_es.sh

ES_HEAP_SIZE=2G bin/elasticsearch -Des.config=${TDIR}/etc/elasticsearch.yml

The viewer would now start (the capture and viewer process were already running but had gracefully killed them). Here are the commands to start each process based on the default installation criteria.

$ sudo nohup /data/moloch/bin/run_es.sh
$ sudo nohup /data/moloch/bin/run_capture.sh &
$ sudo nohup /data/moloch/bin/run_viewer.sh &

Sessions page screen-shot after capturing some traffic, not including session listing:

moloch-graph-thumb

Stats page screen-shot:

moloch-stats-thumb

I noticed the mention of two plugins to keep tabs on the elasticsearch memory usage and to maintain session data. This is pretty important as I determined if you remove PCAP and the session data remained, think metadata, users that attempted to drill-down on the aforementioned session data for the missing PCAP would cause the viewer process to die. In my case, I setup Putty to tunnel my connection to the locally listening plug-in interfaces and delete the offending session data:

moloch-putty

ElasticSearch maintenance screenshot located at http://127.0.0.1:9200/_plugin/head/ after tunneling via Putty. I was able to drop the session via this interface.

moloch-head-thumb

Node statistics screen-shot accessed at http://127.0.0.1:9200/_plugin/bigdesk/ after correctly configuring Putty. Note that we want to keep an eye on the heap memory to ensure it does not approach the maximum specified value. There are many more statistics not shown in this screen-shot.

moloch-bigdesk-thumb

Here’s a Youtube video featuring Moloch in actions. As usual, if you have trouble installing or running Moloch, please leave a comment below, and do not forget to check out the Moloch FAQ.

Posted in network, security | Tagged , , | Leave a comment