A story of rootkit attack on a linux server

Yesterday, got a request from a friend to check his linux server for any security compromise.

Hmm. Linux is a very secured one. But, it is upto the admin who maintains it. We have doors and locks. We have to make sure that the doors are closed and locked.🙂

Checked the server. I felt that it was cracked.😦

“last” says that
saravana pts/2        78.96.162.69     Thu Dec  3 03:22 – 03:23  (00:00)

geoip sites said that.
IP Address:     78.96.162.69
Country:        Romania romania
Country code:   RO (ROU)
Region:         Vrancea
City:   Focsani

All histories before dec 3 were cleared.

After some checkup, found the culprit.

Found that some cracker entered on Dec 3 as user saravanan.

He runs so many ssh daemons and do a lot of port scans to other servers.

ps aux | grep ssh | wc -l
450

found so many processes like ./ssh 270 and ./x

there should not be any separate process like ./ssh and ./x

so searched in /home/saravanan.
hahaha. here, the thief lives.

found some hidden folders as “.a” “.c” “.d” and “h” in /home/saravanan

In the folders there are so may scripts to call ssh and do portscan
and try various username/password by using bruteforce algorithm.

This made our server to scan many servers and they mark us spam.

I deleted the .a .c .d h folders.

killed all ssh processes and requested for a re-installation of the server.

The root cause of the issue are

1. very very very weak password for user saravanan.
2. there is no firewall
3. no powerful logging methods
4. no limits on no of processes/diskspace

Chennai Linux User Group is so helpful. When I asked suggestions regarding this case,

Arun SAG:
Disable password login to your remote server, use ssh keys instead.

If possible configure your system so that  it stores logs in another system.

Find out the open ports and check for any know vulnerabilities .

Arun Khan:
Looks like somebody guessed your password.

Somebody local with access to a machine in RO could have come in and
cracked your system and make it appear that the attacker is physically
from RO.

For RPM based distro you can verify the integrity of your files
with –verify option; assuming the cracker has not messed with your rpm
data base.  I don’t know the Debian equivalent.

You may luck out with the cracker just wanting to break in and if your
file integrity checks out in your favor.

Rather than spinning your wheel figuring out what is b0rked;   a fresh
install would save you time.  After a fresh install create a `ro’
signature db of your files (aide/tripwire) and keep a copy in a
separate location.  aide/tripwires can be configured to check integrity
of the installed files v/s in the signature db.

Raja Subramanian
:
Setup an iptables based connection limiting rule so that you deny more
than 3 ssh connects per minute from any IP address.

Ther are more intelligent scripts like fail2ban, but I usually find a
simple 2 line iptables connection limiter sufficient to stop bots from
brute forcing your passwords.

Thanks folks.

2 thoughts on “A story of rootkit attack on a linux server

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s