Kieran Jacobsen 23/8/11 Kieran Jacobsen 23/8/11

Automating Nmap analysis with PowerShell

Nmap is one of the best tools in a sysadmin's toolkit; this powerful tool allows us to quickly determine what computers and devices are on our network, what software and operating systems are running.

In most environments, esp. when there are a large number of servers and workstations in quite a number of subnets, its handy for system administrators to be able to easily find a free IP address.

There have been several times when various security managers have requested to know the following items:

All servers/workstations that are up
Are the following services running: SSH, Telnet, FTP, HTTP, HTTPS, RDP, SMTP
Is there anonymous FTP?
Is there anonymous FTP uploads?

Nmap is obviously the tool to complete the task to find out this information. If we could automate this, then we could easily generate reports for upper management!

Thankfully, the guys over at SANS have already completed most of the work for us. In the post: PowerShell Script To Parse nmap XML Output, the provide a script which gets Nmap XML output and makes it into a format that allows any powershell user to manipulate the results using all the usual commands (format-table, format-list, where-object, select-object).

I developed a simple script to find IP addresses that were in use and provide a simple output that any system administrator or service desk operator could read. The script is simple, and does the following tasks for each subject listed in an array:

Make a user friendly filename
Run nmap to perform a number of ICMP and TCP scans to find servers that are up (I recommend TCP SYN scanning on top of ICMP Ping to ensure you find firewall protected servers and workstations)
Parse the results of nmap and put them into a file in a more friendly format

The script looks like this:

nmap Command breakdown:

-PS20,21,22,23,25,3389,80,443,8080 is a TCP SYN Ping sweep of the subnet of ports 20, 21, 22 etc

-PE is ICMP ping (usual ping command)

-R is perform a reverse DNS look up

--dns-servers is specifying all of our DNS servers (incase you have reverse lookup zones across differing DNS servers)

-p 20,21,22,23,25,3389,80,443,8080 we want to scan these ports for possible reports later on

-oX $nmapfile --no-stylesheet outout the results to the filename and don't use a XML sylesheet

-A enable all advanced options

-v we want verbose output for reporting.

We end with the subnet we want to scan.

Kieran Jacobsen 22/8/11 Kieran Jacobsen 22/8/11

MailScanner on Centos

This is a simple guide to building an email gateway which will perform anti-spam and anti-virus filtering prior to delievering email to its intended destination.

I also make use of a sendmail milter to verify the recpients of email messages are valid, and drop the messages if the recpient is found to be invalid. Recipient filtering not only reduces the amount of work that the gateway has to do (as it doesn't perform any anti-spam analyasis etc on the message) but reduces the load on the destination server(s) and protects them.

The milter works by simply connecting to the destination server and testing if it accepts the recipient address prior to accepting the rest of the email body from the machine which is connected to it. If you use Exchange as your destination server, make sure it doesn't accept invalid recipients and returns an NDR which is the default. If the recipient isn't valid, the miltor returns to the SMTP server which is attempting to send email to it that the mail box is full or invalid.

Installation Steps

Install Centos
Just follow normal install
Install Updates
yum update
Install Webmin
If you like to install and update Webmin via RPM, create the /etc/yum.repos.d/webmin.repo file containing:
[Webmin]
name=Webmin Distribution Neutral
baseurl=http://download.webmin.com/download/yum
enabled=1
You should also fetch and install my GPG key with which the packages are signed, with the command:
rpm --import http://www.webmin.com/jcameron-key.asc
You will now be able to install with the command :
yum install webmin
All dependencies should be resolved automatically.
Install Mailscanner
export MAILSCANNER_CREATE_TMPFS=1
wget http://yum.fslupdate.com/fsl-beta/fsl-beta.repo -O /etc/yum.repos.d/fsl-beta.repo
yum -y groupinstall MailScannerGold
export PERL5LIB=/opt/fsl/lib/perl5
chkconfig MailScanner on
yum update
Configure Mailscanner
vi /etc/MailScanner/MailScanner.conf
Install Sender/Recipient verification sendmail milter
yum install sendmail-devel
yum install sendmail-cf
yum install libmilter
tar xzvf smf-sav-1.4.0.tar.gz
cd smf-sav-1.4.0
make
make install
Configire address verification milter
Configure Sendmail
Do what ever forwarding and routing you need to configure.

Add the following lines to sendmail.mc above the MAILER(smtp)dnl like
INPUT_MAIL_FILTER(`smf-sav', `S=unix:/var/run/smfs/smf-sav.sock, T=S:30s;R:4m')dnl
Configure Startup scripts to include milter
We need to modify the MailScanner init script at \etc\inif.d\MailScanner to ensure that the process that performs the address verification is started before sendmail and mailscanner.
start)
...
    daemon /usr/local/sbin/smf-sav
...
stop)
    if test "x`pidof smf-sav`" != x; then
    echo -n $"Stopping $prog: "
    killproc smf-sav
    echo
...

Kieran Jacobsen 21/8/11 Kieran Jacobsen 21/8/11

PowerShell RoboCopy Wrapper

So I find myself copying files from point A to point B on a regular basis, so much so, that I find myself using robocopy in a Windows scheduled task. The problem with this approach, is that there isn't many easy ways of working out if it worked, or worse, if something bad has happened!

I bashed out a script several years ago, and finally updated it to make use of the template that I posted up previous.

There are a few things to note with the script:

I always check if the source and destination exist, and report an error if they don't
My scripts always use the /MIR command, and limit retries to 3
Robocopy return codes:
0: Completed sucessfully, but didn't copy any files
1: Completed sucessfully and copied files
rest: some error has occured

The bulk of the code is:

#robocopy task
$result = robocopy $source $destination /MIR /R:3

#robocopy sucess is:    0: no errors, but no copying done
#                       1: one or more files were copied sucessfully

#check there were no errors in the robo copy
if ($lastexitcode -gt 1)
{
   if ($lastexitcode -eq 0)
   {
       $messagebody = "Robocopy completed but no files copied! `n"
       foreach ($x in $result) {$messagebody = $messagebody + $x + "`n"}
       send-email $messagebody $false
   } elseif ($lastexitcode -eq 1)
   {
       #do nothing, it worked
   } else {
       $messagebody = "Robocopy returned an unknown error! `n"
       foreach ($x in $result) {$messagebody = $messagebody + $x + "`n"}
       send-email $messagebody $false
       exit 3
   }
}

The entire script is in the samples section, here.

Kieran Jacobsen 5/7/11 Kieran Jacobsen 5/7/11

Snort on Ubuntu 11.04

Previously I had been running insta-snorby, but after the 100th time of either ruby, ruby on rails, snorby's job or some other failure, it was decided to build a new system, this time where all the parts were easily supported and UP-TO-DATE!

The final result was Ubuntu Server 11.04, with the latest versions of Snort, Barnyard2, PulledPork and SnortReport.

I tried very, very, very hard to also add in Snorby to the mix, and I managed to get its cache jobs to run for a few hours, then they would die with no useful error messages. It doesn't help that some of the official documentation points you to an old repository, it also does help that the GEMs it makes use of are highly unstable.

I also looked at Sguil and its framework, its nice but i dislike the need to install a client, and massively change the effiency of this Snort deployment.

It should also be added that both Snorby and Sguil fail at one thing, having documentation. You need accurate documentation, and also need documentation that isn't 5 years old. SnortReport is old, but the documentation was perfect, not that i needed it.

Anyway, here is how I did it.

1) Follow the Installing Snort on Ubuntu 10.04 guide at http://www.snort.org/docs.

I didn't install Snort to /usr/local/snort, I instead put everything in the default paths, its just a lot less work.

2) Installed PulledPork according to the documentation

3) To start snort and barnyard2:

sudo ifconfig eth1 up

sudo snort -D -u snort -g snort -c /etc/snort/snort.conf -i eth1 --pid-path /var/run/snort

sudo barnyard2 -c /etc/snort/barnyard2.conf -G /etc/snort/gen-msg.map -S /etc/snort/sid-msg.map -d /var/log/snort -f snort.u2 -w /var/log/snort/barnyard2.wald -D --pid-path /var/run/barnyard2

4) To update rules:

sudo perl /usr/local/pulledpork/pulledpork.pl -c /etc/snort/pulledpork.conf

There really isn't anything else you need to do.

I have included various config files in the brain samples section:

Kieran Jacobsen 14/6/11 Kieran Jacobsen 14/6/11

Windows Server 2008 R2 + LVS with Direct Routing and Windows Firewall

For those of you who are in the need for an IP LoadBalanacer, and do not want to pay for an F5, check out the LVS project.

We recently set up a number of LVS balanced pages, and quickly came into difficulty in selecting thr routing method used. We struggled to find documentation, and were told to use the NAT technique, something that we were not happy with. The lack of documentation was also not helped due to the fact we wanted to load balanced Windows 2008 R2 servers running IIS 7.5.

We managed to work out how to set up both the Linux side of the fence (the machines running LVS) and then what to do on the Windows Servers being balanced. We also managed to leave the Windows Firewall on!

Once you have your LVS setup running. Perform the following steps to

1. Perform the standard configuration using what ever method you like (Piranha the web interface is brilliant for this) and ensure you select "Direct Route"

2. Restart Pulse service

3. Add the Loop back adapter to each Windows machine as specified at DR and LV Tun Clusters

4. You do not need to disable the Windows Firewall

5. Setup weak host send and recieve as specified at the loadbalancer blog.

Kieran Jacobsen 11/6/11 Kieran Jacobsen 11/6/11

Powershell Script Template

Just a quick post to cover layout/format of all PowerShell scripts I write.

This default template ensures that a sufficient amount level of of quality is ensured in all scripts. There are several things you will notice:

All scripts start with param definitions, which include items for the functionality for that script (both mandatory and option) as well as variables used to support features other basic functions
Email alerting on success of failure with option to turn off or on both, either, or no email alerts
I use exit calls, I know this is probably considered bad, but as you can see, it makes it easier to troubleshoot

And here it is (you can download it here):

# ==============================================================================================
#
# Microsoft PowerShell Source File -- Created with SAPIEN Technologies PrimalScript 2009
#
# NAME: <name of script which will be helpful>
#
# AUTHOR: <author>
# DATE : <date>
#
# COMMENT: <Step through what this script does
#
# ==============================================================================================

#<place comments used to assist in the calling of the script

param(
   #<application specific params>

   #<email and alerting params>
   [Parameter(Position=3, Mandatory=$false, HelpMessage="Specify email alert recipient address")]$recipient="[email protected]",
   [Parameter(Position=4, Mandatory=$false, HelpMessage="Specify email alert from address")]$from="[email protected]",
   [Parameter(Position=5, Mandatory=$false, HelpMessage="Specify smtp server name/ip")]$smtpserver="smtp.server.com",
   [Parameter(Position=6, Mandatory=$false, HelpMessage="Alert on sucessfuly copy")]$emailonsuccess=$true,
   [Parameter(Position=7, Mandatory=$false, HelpMessage="alert on failure of copy")]$emailonfailure=$true,
   [Parameter(Position=8, Mandatory=$false, HelpMessage="alert email subject")]$emailsubject="<change to be helpful subject>"
)

#
#variable declarations


function send-email ($body, $success)
{
   #incase we are troublshooting, output the body here
   $body

   $error.clear()
   #if it is sucessful
   if ($success)
   {
       #if we want a successful emails
       if ($emailonsuccess)
       {
           $subject = $emailsubject + "- Success"
           Send-MailMessage -To $recipient -From $from -SmtpServer $smtpserver -Subject $subject -Body $body -bodyashtml
       }
   }
   else #if unsuccessful
   {
       #if we want emails on failure
       if ($emailonfailure)
       {
           $subject = $emailsubject + "- Failure"
           Send-MailMessage -To $recipient -From $from -SmtpServer $smtpserver -Subject $subject -Body $body -bodyashtml
       }
   }

   if ($error)
   {
       "Unable to send an email"
       exit 666
   }
}

#
# If we expect a crash/error here is an example (this is using $error but it could be try catch or based on $lastexitcode for external calls
#
$error.clear()
#do something bad
somethingbad()
if ($error)
{
   send-email "<sad>" $false
   exit 7
}

#email that we got here successfully
send-email "<HAPPY>" $true