BotHunter is an application designed to track the two-way communication flows between internal assets and external entities, developing an evidence trail of data exchanges that match a state-based infection sequence model.  BotHunter consists of a correlation engine that is driven by a customized and augmented release of Snort version 2, which tracks the underlying actions that occur during the  malware infection process: inbound scanning, exploit usage, egg downloading, outbound bot coordination dialog, outbound attack propagation, and malware P2P communication.  The BotHunter correlator then ties together the dialog trail of inbound intrusion alarms with those outbound communication patterns that are highly indicative of successful local host infection.  When a sequence of evidence is found to match BotHunter's infection dialog model, a consolidated report is produced to capture all the relevant events and event sources that played a role during the infection process.  We refer to this analytical strategy of matching the dialog flows between internal assets and the broader Internet as  dialog-based correlation (patent pending).  

BotHunter is available free for both experimental operational use and to help stimulate research in understanding the life cycle of malware infections.

Hardware Requirements

OS and Software Requirements

Communication Requirements

Where to Install BotHunter

For site-wide network monitoring, your target platform should have promiscuous-mode access to broadcast LAN traffic via port mirroring (e.g., Cisco Switched Port Analyzer (SPAN), 3COM Roving Analysis Port (RAP), etc.).  Ideally, your machine should be attached to a monitoring position on an internal network egress point to observe successful connection flows.

We strongly recommend that you place BotHunter behind your firewall. It does not need to monitor incoming packets that are blocked from entry to your net. See BotHunter Behind or In Front of Firewall for configuring BotHunter when the tap is in front of a firewall.

The following is a summary of the minimum steps necessary to install, configure, and start BotHunter, in its default configuration for live traffic monitoring.  This installation procedure should be performed by the root user.  You will also need to know the IP address netmask of the network you wish to protect, and the IP addresses of your email and DNS servers.

For Unix-based systems, BotHunter's root-phase installation process will detect a prior installation to the selected non-privileged user account and offer to rename the prior installation directory (which can later be safely removed). If you decline the rename, the installation will terminate. The network information from the prior installation (home net, SMTP & DNS servers, and network interface) will become the defaults for the current installation process, but any other uniquely-set (non-default) configuration information will need to be re-applied.

Software Installation Procedure

While installation requires root privilege, BotHunter does not require root privilege to run. Instead, this installation creates a nonprivileged user account that runs BotHunter.

Note: you may type '?' at any prompt for a detailed explanation of what is expected. 

      Read the EULA and if acceptable click YES.

3.   Confirm that you wish to perform this root install.

4.   Optional: You are prompted to install Tor if it has not been installed previously. BotHunter may be configured to use Tor to interact anonymously  with the BotHunter repository services.

5.   Indicate the new non-privileged user account with which you wish to install BotHunter (default user account = cta-bh).    BotHunter will then install dependent packages. If you choose to install BotHunter over a preexisting user account, this account must use csh(1).

6.   Enter your Trusted Network Mask:  Provide a (comma separated) local network mask list, plus the IP addresses of all external NetBIOS shares with which your internal machines are allowed to communicate.

     example: 192.168.1.0/24,10.10.0.10/16

7.   Enter the (comma separated) IP addresses of the email server(s) used by systems inside your network.

8.   Enter the (comma separated) list of DNS servers used by systems inside your network.

9.   Enter your network interface that BotHunter will use to monitor your network.

10. Indicate whether you wish BotHunter to start automatically on system boot. If you answer "yes", a default configuration will be created for the non-privileged user and you will be prompted to start the BotHunter process. If the default configuration is satisfactory, you may start BotHunter and skip the user configuration procedure.

11. Optional: As a last step, you may now set the non-privileged user's password. For example:

       root% /usr/bin/passwd cta-bh

User Configuration Procedure

You must now complete the user configuration phase of the BotHunter installation procedure.   This step is performed as your installed user target, e.g., cta-bh (not as root).

12. su to the user account that you created during the BotHunter installation

       root% su -l cta-bh

13. To run BotHunter in its default configuration, use the BotHunter shell alias:

       cta-bh% BotHunter

First-start Default Configuration: upon the first invocation of BotHunter, with no configuration established through root installation (i.e., by not selecting the on-boot option), the default configuration information will be displayed before BotHunter is started. The default configuration of BotHunter will inherit the parameters that were submitted during your root installation.

If you wish to view, with the option to change, the BotHunter configuration, you must add the "configure" option to the BotHunter command-line arguments:

       cta-bh% BotHunter configure

See Section Configuring BotHunter for details regarding how to customize the BotHunter runtime configuration. At the configuration prompt, you may type 'done' when you have completed any configuration changes and are ready to proceed. You will then be prompted to start BotHunter or return to the command prompt. If you select 'no', you can later start BotHunter using the BotHunter command. If you type 'yes', BotHunter will start itself and return control back to the command prompt.

14. How to manage BotHunter  

• - If you prefer to manage BotHunter through the GUI, refer to the
•    BotHunter GUI User Guide for details.
• - If you prefer to manage BotHunter in console mode, please read 
  
•    section Operating  BotHunter for information on how to manage BotHunter.

Express Mode Setup

The BotHunter configuration menu will be in 'express mode' by default, which provides the basic configuration options appropriate for most users.  More detailed configuration control is available by switching to 'Custom' configuration mode.   The following summarizes BotHunter's express mode configuration.   If you wish to enter custom mode, please follow the online help using the '?' response to all configuration prompts.

Input Parameters:  Select option '1' to configure logs

Select which input source type you prefer to use when operating BotHunter:

1. LIVEPIPE: (Recommended) Sort dialog events are redirected from the standard output of a command executed by BotHunter as needed (e.g., a wrapper script that starts Snort or a direct Snort invocation).  This is the default mode provided during installation.  Use the alias 'BotHunter' command to start BotHunter in this mode:

cta-bh% BotHunter  [gui]

The 'gui' command line option is used to invoke the BotHunter graphical user interface.  The GUI allows you to start, shut down, and monitor the runtime operation of BotHunter, view BotHunter infection profiles, update the BotHunter ruleset, and receive BotHunter announcements from SRI.   For more information on using the GUI, see the BotHunter GUI Guide.

2. LIVEFILE:  Snort dialog events are written to a  log file, and read live by the BotHunter Correlator.   Note: before you can operate in LIVEFILE mode, you must first direct Snort to produce a continuous alert stream into a log file, e.g.  See section Operating BotHunter in Live File Mode for more details.
 
3. BATCH:  Snort dialog event logs can be stored, shared, and rerun through BotHunter in batch mode.   For example, you may test BotHunter batch mode operations by downloading and selecting the dialog event logs posted to the BotHunter Sample Malware Analyses page (specify 192.168.0.0/16 as the trusted network when using these samples).  By default, batch alerts are not forwarded to the BotHunter repository.  See section Operating BotHunter in Batch Mode for more details.

4. INLINE:   BotHunter is invoked as an element in an externally-created pipe sequence, and as such  BotHunter terminates when the standard input stream terminates.  The user is therefore responsible for restarting the pipe sequence should any element in the sequence terminate.

Snort Parameters: Select option '2' to configure BotHunter's Snort configuration

Select option '2' to reconfigure the Snort parameters prompted for during the root installation phase.

Output Parameters: Select option '3' to configure infection reporting

Select option '3' to enable the SRI  Infection Reporting and Ruleset Updating Service. Enabling this service allows us to regularly upgrade your fielded BotHunter with the latest malware threat intelligence, including newly discovered botnet control servers and drop sites, malware-related DNS query identification, and the latest malware detection signatures.   To utilize BotHunter's dynamic update services, you must enable the anonymous infection reporting services of BotHunter.  Express mode allows you to modify the repository updating and reporting parameter, while Custom mode allows you to modify all parameters.

Anonymization Parameters: Select option '4' to configure output

This option allows you to configure the anonymization policy used by the BotHunter Infection Reporting and Updating Services.

Diagnostic Parameters: Select option '5' to configure diagnostics

This option is not available in 'express mode.'  In 'custom mode' this option allows you to configure BotHunter's diagnostic reporting levels and select the diagnostic log directory.

NetQuery Parameters: Select option '6' to update service

This option allows you to define the interval checks used for managing BotHunter's dynamic updating and notification services. 

File and Directory Structure

When BotHunter is first invoked, it creates a set of configuration files inside the directory in which it is run.  These files establish the keys, certificates, logs, and configuration settings that are used when the BotHunter process is run.  The following files are created in the local directory from which BotHunter in invoked:

If you select NOT to operate BotHunter in the recommended configuration, your next prompt will ask you to select an input source for Snort alerts.  You may choose input source "OPTION 2" to have BotHunter process alerts from a live Snort log.  This method allows you run Snort externally from BotHunter, but you must assume the burden of restarting Snort when it fails and managing disk storage space.

In live file mode, BotHunter does not terminate once it reaches the end of the specified input file, but waits for more data to be appended to the file.  Also, BotHunter continuously monitors the path associated with the file for indications that the inode associated with it has changed (file size shrinks, file suddenly not readable, or EOFs are being read, but the file's modification date keeps increasing).  When BotHunter reaches the end of the current file and has detected an inode change, it will re-open the specified path and begin processing alerts from the new inode. Note that BotHunter deals with only a single file name path and does not do pattern matching on changing file names over time.

In addition, to reduce the potential for duplication of alerts in the repositories, BotHunter ignores alerts with timestamps earlier than two hours before the BotHunter process was started.  Thus, if BotHunter is provided a Snort log that has been accumulating data for more than two hours, it processes only the last two hours' worth of alerts (and, once it catches up, continues to process alerts as described above).

One way to invoke Snort to create a live alert log is as follows (the script runsnort.csh is created in user cta-bh's home directory during the self-installation process):

Follow the prompts to set address anonymization policy, input and output file options, and anonymous publication options.  You may enter '?' at any prompt for further information regarding options.

Operating BotHunter in Batch Mode

Input source "OPTION 3" allows you to select a Snort file to process in batch mode.  Using BotHunter in batch mode (i.e., processing a previously generated Snort log) is the same as for real time, except that you provide the Snort log file as an additional argument on the command line:

cta-bh% mkdir ~/BotHunter/BATCH_CONFIG
cta-bh% cd ~/BotHunter/BATCH_CONFIG
cta-bh% java -jar ../botHunterInstall.jar configure
<Select Input Source "OPTION 3">

When configuring BotHunter in batch mode, you must ensure that you set the Trusted Network mask (select Option '1' of the configuration panel) to match the target network of the batch alert set.  Only one Snort log file can be processed at a time.  The command does not return control until the run completes.  The batch run creates the same diagnostic files as a live run.

Operating BotHunter in Inline Mode

Creating Multiple Runtime Configurations

BotHunter allows you to create multiple runtime configurations using separate configuration subdirectories.  Each subdirectory may be used to establish a different runtime configuration that will operate in either of three potential operating modes.  

BotHunter can be configured to run in multiple input modes and to operate with various output and diagnostic parameters.  The BotHunter configuration menu is shown when you run BotHunter for the first time or by invocation of the configure command line option as follows:

With each subsequent invocation of the BotHunter jar file, BotHunter will use the configuration that has been established within your local directory during the initial invocation of the system.  For example, if you wish to maintain two instances of BotHunter, one instance for batch file testing and one instance for LIVEPIPE operation, we recommend that you create two separate subdirectories and perform two independent installations of BotHunter:

Example: create one directory for LIVEPIPE real-time monitoring:
      
cta-bh% cd /home/cta-bh/BotHunter
cta-bh% mkdir LIVEPIPE_CONFIG
cta-bh% mkdir BATCH_CONFIG

You may now set up both of these directories independently, by changing directory (cd) into each directory and then invoking the botHunterInstall.jar file:

cta-bh% cd ../LIVEPIPE_CONFIG
cta-bh% java -jar ../botHunterInstall.jar configure
<follow prompts to establish livepipe runtime configuration>

To establish the BATCH MODE configuration:

cta-bh% cd ../BATCH_CONFIG
cta-bh% java -jar ../botHunterInstall.jar configure
<follow prompts to establish batch mode runtime configuration>

From this point forward, you may select which configuration you wish to run simply by entering the appropriate subdirectory (e.g., enter directory BATCH_CONFIG for processing your batch file tests).

Revising Your Runtime Configuration

You can reconfigure an instance of the runtime configuration by adding the directive 'configure' to the command line invocation of the BotHunter jar file.

Example:

cta-bh% cd /home/cta-bh/BotHunter/BATCH_CONFIG
cta-bh% java -Xmx104m -jar ../botHunterInstall.jar configure
 <revise your configuration using configuration prompts>

Shutting down BotHunter

If you are operating BotHunter using the GUI, you may shut down the current instance by selecting the "Shutdown" button.  If you are operating BotHunter in console mode and are using the default configuration, as described above, you may shut down BotHunter through a command line argument:

cta-bh% BotHunter shutdown

If you have created an alternate configuration instance of BotHunter (see Operating BotHunter), you must first change directory (cd) to your alternate BotHunter configuration directory, for example:

cta-bh% cd /home/cta-bh/BotHunter/LIVEFILE_CONFG
cta-bh% java -jar ../botHunterInstall.jar shutdown

Shutdown may take a few minutes until the signal is processed and buffers are flushed.  A final status display may appear as part of the output from the shutdown command.

Using the Status Option

To check the status of a live running BotHunter instance, you may use the 'status' command line option. If you are using the "recommended configuration", simply add the "status" command line argument to the BotHunter invocation:

cta-bh% BotHunter status

If you are not running the recommended configuration, you must first cd to the BotHunter configuration directory, for example:

cta-bh% cd /home/cta-bh/BotHunter/LIVEFILE_CONFIG

Next, re-invoke the BotHunter jar file, adding the command directive 'status' to the invocation:

cta-bh% java -jar ../botHunterInstall.jar status

BotHunter will produce an operational status summary similar to the following:
 
[cta-bh@sensorX LIVEPIPE_CONFIG]$ BotHunter
2008/08/24 21:48:51 PDT Significant: Diagnostic log is now
CTA_BotHunter/logs/CTA_BotHunter_20080924_214851.log

Started CTA_BotHunter process: 2008/09/24 21:48:50 PDT
[cta-bh@sensorX LIVEPIPE_CONFIG]$ botHunter status

CTA BotHunter 1.0.3 status #1 as of 2008/09/24 21:49:13 PDT
  Process elapsed time:               0 00:40:20
  Memory usage:                       28553 Kbytes
  Input events read:                  931
  Input events parsed:                931
  Local text BotHunter profiles:      2
  Messages sent to repository:        2
  Sensor connected to repository:     true
  Most recently seen author ID:       999999ffffff
  Most recently observed ID:          101010101
 
Latest information from the repository at 130.107.10.11:
message text: BotHunter v1.0.3 is now available.

CTA BotHunter: Process is active: waiting for BotHunter reports.
[cta-bh@sensorX LIVEPIPE_CONFIG]$

How to Validate BotHunter Using Sample Alert Files

These sample alerts are produced using a TRUSTED NET = 192.168.0.0/16, and can be processed as follows:

1. Create a batch mode configuration instance of BotHunter

cta-bh%  mkdir ~cta-bh/BotHunter/BATCH_SAMPLES
cta-bh%  cd ~cta-bh/BotHunter/BATCH_SAMPLES/
cta-bh%  java -jar ../botHunterInstaller.jar configure

- select Input Option '1' to create a batch mode configuration

2. To exercise the sample files (assuming the sample files have been moved to file sample.log):

The BotHunter profile will be stored in file sample.log_botHunter.txt

BotProfile Format

BotHunter produces an Infection Profile when it encounters a machine inside the Trusted Network address space that exhibits a pattern of dialog exchanges that match the correlator's internal infection life cycle mode.  Infection profiles are stored in the local text output file (by default  BotHunterResults_%dt.txt) as defined within the BotHunter configuration Panel (see section Configuring BotHunter, subsection Output Parameters). 

All BotProfiles consist of three sections: the profile header, forensic evidence, and packet selection instructions.   The profile header consists of the following fields:

Score:            2.6 (>= 0.8)
Infected Target:  192.168.1.41
Infector List:    77.102.0.196
Egg Source List:  77.102.0.196
C & C List:       210.245.211.11
Peer Coord. List: <unobserved>
Resource List:    <unobserved>
Observed Start:   07/25/2008 05:03:53.171 PDT
Report End:       07/25/2008 05:10:43.628 PDT
Gen. Time:        07/25/2008 05:10:43.628 PDT

INBOUND SCAN
    <unobserved>
    
EXPLOIT
    77.102.0.196 (05:03:53.171 PDT)
       event=1:299913 {tcp} E2[rb] SHELLCODE x86 0x90 unicode NOOP
          135<-3083 (05:03:53.171 PDT)
    
EGG DOWNLOAD
    77.102.0.196 (9) (05:03:57.087 PDT-05:03:58.131 PDT)
       event=1:1444 (3) {udp} E3[rb] TFTP GET from external source
          1029->69 (05:04:13.135 PDT)
          2: 1028->69 (05:03:57.087 PDT-05:03:58.131 PDT)
       -------------------------
       event=1:2008120 (3) {udp} E3[rb] ET Policy Outbnd TFTP Read 
          1029->69 (05:04:13.135 PDT)
          2: 1028->69 (05:03:57.087 PDT-05:03:58.131 PDT)
       -------------------------
       event=1:3001441 (3) {udp} E3[rb] TFTP Get .exe from external src 
          (05:04:13.135 PDT)
          2: 1028->69 (05:03:57.087 PDT-05:03:58.131 PDT)
       
C and C TRAFFIC
    210.245.211.11 (05:04:25.309 PDT)
       event=1:2003603 {tcp} E4[rb] ET TROJAN W32.Virut.A joining IRC
       Channel 1032->65520 (05:04:25.309 PDT)
       
PEER COORDINATION
    <unobserved>
    
OUTBOUND SCAN
    77.102.0.196 (05:03:53.386 PDT)
       event=1:52123 {tcp} E5[rb] Registered Free Attack-Response 
       Microsoft cmd.exe banner 1027->707 (05:03:53.386 PDT)
       
ATTACK PREP
    <unobserved>
    
DECLARE BOT
    210.245.211.11 (3) (05:04:24.848 PDT-05:10:43.628 PDT)
       event=1:3000014 (3) {tcp} E8[rb] BotHunter Known C&C Server
       (International) 3: 1032->65520 (05:04:24.848 PDT-05:10:43.628 PDT)

tcpslice 1216987433.171 1216987843.629 inputFile.tcpd | tcpdump -r - \
         -w outputFile.tcpd 'host 192.168.1.41' 

BotHunter has additional capabilities available through the "custom" configuration mode. This section documents some of the more frequently requested options. To enter "custom" configuration mode:

1. Enter BotHunter user configuration mode, using the BotHunter shell alias:

   cta-bh% BotHunter configure

2. Enter custom configuration mode by entering "custom" at the configuration prompt:

     Enter the number of the section to alter,
     "?"       for help on this prompt,
     "custom"  to switch to custom configure mode.
     "reset"   to restore the configuration to the "factory defaults",
     "abort"   to abort the installation and exit, or
     "done"    to save this new configuration (default: done): 
   

BotHunter Behind or In Front of Firewall

If you must install BotHunter in front of your firewall, then enter "custom" user configuration mode and modify section "1" (Input Parameters) and hit enter to use the default for all prompts until you reach the prompt:

  Are the packets examined by snort from behind a firewall (default: true): 

Infection Log Roll-over

BotHunter has the ability to automaticly roll-over the infection log to a new file based on time, size, or number of reports. To enable this feature, enter "custom" user configuration mode and modify section "3" (Output parameters) and hit enter to use the default for all prompts until you reach the prompt:

  Enter the name of the text output log file (default:
    botHunterResults_%dt.txt): 

Email Notification

BotHunter has the ability to send infection reports via e-mail. This feature is independent of any other infection reporting mechanism (e.g., it can be used in addition to, or instead of, the GUI). To enable this feature, enter "custom" user configuration mode and modify section "3" (Output parameters) and hit enter to use the default for all prompts until you reach the prompt:

  Enter the mail "to" destination (default ""): 

ArcSight CEF Alerts

BotHunter has the ability to generate infection reports in the ArcSight Common Event Format. This feature is independent of any other infection reporting mechanism (e.g., it can be used in addition to, or instead of, the GUI). To enable this feature, enter "custom" user configuration mode and modify section "3" (Output parameters) and hit enter to use the default for all prompts until you reach the prompt:

  Enter the name of the file to receive ArcSight Common Event Format
  alerts (default: none): 

Saved Snort log

When run in live-pipe mode (input mode "1"), BotHunter has the ability to save the Snort alert text logs used as input to the dialog correlation step and to roll over those log files based on time or size. Enter "custom" user configuration mode and modify section "1" (Input Parameters) and hit enter to use the default for all prompts until you reach the prompt:

  Enter the log file name to receive a local copy of the raw input
  data (use "none" for no local copy, default none): 

The following lists some of the more notable changes from the prior (1.0.2) release:

• Added ability to change home-net, DNS, and SMTP address lists after root installation.
• Added behind-firewall option to installer (see BotHunter Behind or In Front of Firewall).
• Added infection log roll-over options (see Special Features).
• Added optional Email delivery of infection reports (see Special Features).
• Added optional ArcSight CEF Alert output (see Special Features).
• BotHunter may now be permanently (and successfully) installed from the BotHunter Ubuntu LiveCD.
• Added desktop icons to start BotHunter GUI for Linux and Mac OS X.
• Added test for, and installation of, patch when required.
• Performs name-to-address lookups where IP addresses were previously required.
• Attempts, within the confines of Java, to display which network adapters are available.
• Allows for multiple input file arguments in batch mode.

The BotHunter team gratefully acknowledges those increasingly fewer U.S. funding agencies that are actively supporting new research in information security.    We especially thank Cliff Wang at ARO for his support of the Cyber-TA project and  BotHunter.