It was bound to happen - suspected hack

[Dustin Decker]

> -----Original Message-----
> From: kclug-bounces at kclug.org [mailto:kclug-bounces at kclug.org] On Behalf
> Of Dave Hull
> Sent: Thursday, October 21, 2004 11:21 AM
> To: kclug at kclug.org
> Subject: RE: It was bound to happen - suspected hack
> 
[snip]
> What kind of storage space is required for this? How long to do you keep
> all
> that traffic? That sounds impressive.

Well, it depends on a number of factors - primarily the speed of the link
being monitored, and the utilization of the link as well.  Monitoring 10/100
is pretty well sustainable without packet loss.  Moving into gigabit is
another world entirely.

I have been known to use the client components from the SHADOW IDS project
to log traffic into binary tcpdump format, on one hour boundaries.  Thus, I
wind up with files like:

-rw-r--r--    1 ids     ids         3043 Oct 21 02:00 tcp.2004102101.gz
-rw-r--r--    1 ids     ids         7076 Oct 21 03:00 tcp.2004102102.gz
-rw-r--r--    1 ids     ids         1456 Oct 21 04:00 tcp.2004102103.gz
-rw-r--r--    1 ids     ids       160916 Oct 21 05:00 tcp.2004102104.gz
-rw-r--r--    1 ids     ids         9574 Oct 21 06:00 tcp.2004102105.gz
-rw-r--r--    1 ids     ids       194180 Oct 21 07:00 tcp.2004102106.gz
-rw-r--r--    1 ids     ids       161428 Oct 21 08:00 tcp.2004102107.gz
-rw-r--r--    1 ids     ids      5623292 Oct 21 09:00 tcp.2004102108.gz
-rw-r--r--    1 ids     ids       638976 Oct 21 09:36 tcp.2004102109.gz

I've a handful of methods to pull the files from remote sensors which I
monitor, etc. and clean the sensor hard drive to ensure I don't run out of
disk space.  I have as of late been moving toward the use of IDABench, which
is based on the SHADOW IDS, which functions nearly identical but is a bit
more current vis-à-vis features.

You can see the file sizes on the snippet of directory listing above.  The
particular sensor this is from is on a DSL link which can sustain 5.99mpbs
inbound, and 384kbps outbound, so storage isn't a huge issue.  What's
important is to pull the files from the sensor frequently enough to ensure
your logging partition isn't getting full.  (<= 24 hours for me, depends on
client.  I have one for which I'm running an hourly pass.)

I can generally store about a month of compressed files on a CD, which is a
good immutable storage mechanism if I need to present them in the future.

Folks who are really interested in going down this road will find Snort 2.1
Intrusion Detection 2nd Edition as useful as the Stevens TCP/IP bible.  It's
from Syngress press, and a host of contributing authors.  'Tis around $42.05
@ MicroCenter.

There are several vendors out there that like to throw around the
buzz-phrase "real time"... don't believe the hype.  In my experience,
anything less than $50K will only get you to within the past half an hour at
best... and you still have false positives (or worse, negatives) to deal
with.

What did Bruce Schneier say?  Oh yeah, "Security is a _process_, not a
_product_."  And much like Neil Stephenson pointed out on /., the hardware
is there, but software is still pretty much shit.

Dustin