ACK! How to fix a compromised system?

[Dave Hull]

On Sat, 19 Apr 2003, Bradley Miller wrote:

> I've got a box that someone put a "toolz" kit on yesterday.  Any ideas on 
> how to cleanse the beast?

As others have said, if you really want to be sure, wipe the box and reinstall 
the OS, offline and get it patched before plugging the NIC back in.

However, if you've got "trustworthy" media, you could bring up the system 
using whatever method your distro provides for emergency recovery and run a 
checksum of everything that's installed on the system.

For example, if you're running RH and you can boot off of "known good" media, 
then you could use rpm to verify your binaries, read the man page for details.

If you can't boot off of a clean kernel, then there's not much hope. I helped 
someone recover a system not too long ago. The box was way out of date and 
whoever tried to compromise it was attempting to install kernel modules which 
were incompatible with the kernel on the system, the mods were too new. As a 
result the system was very unstable and would not stay up for more than a few 
minutes at a time.

I found an old kernel image in the boot directory and patched up lilo.conf to 
use the old kernel image. After rebooting the machine, it came up stable and I 
was able to run an rpm verify (from shrink wrapped RH media) and found the 
usual ls, top, ps, netstat, etc. replaced with trojans, not surprising.

In the end, I recommended that the owner of the box, backup all the data he 
wanted to keep and reinstall a current version of the OS and get it fully 
patched before going back online. It's the only way to be 99% certain you're 
clean.

-- 
Dave Hull
http://insipid.com

Now, done right, we'd acquire lots of ice, acclimate some penguins, and put 
up some sculptures so it looks kind of like Finland; but if nobody claims
not to have a cellphone and you pretend the Indians are Russians, no prob.
-- Steve Nordquist, Re: BBQ for 10th anniv of Linux, 08/21/01