Patrick, I suggest you fire up the server for a few hours and run a PCI scan on it...
qualys.com and several other vendors will do one for free. and/or http://en.wikipedia.org/wiki/Nessus_(software) Also, at some point do a check for rootkit when you are booted from media that is locked for writing (a rescue cd) Good luck. Mark Engelhardt On May 12, 2011, at 5:18 PM, Patrick Litke wrote: > Hello, Hello all! This is my first post to the list serv, after much > encouragement from David Tisdell > > I've taken a few classes with Dave covering Network Security and Linux, and > have been playing around with Linux for a few years now. I wound up buying a > set of cheap hardware for a linux server (both my self and a friend did the > same) to be predominantly used as a VoD box / media server / Samba server. > I'm planning on using mine as a test bed for software development, and > subsequently need it connected to the internet. > > We wound up using his server as a test box for everything. We got Debian > installed, full lamp stack, nagios, webmin, samba, and a few others. > Everything was working fine - we had remote SSH access, full http access > (only over SSL though), remote access to webmin - the works. We did not have > pasword-less logins (ie: using keys) for SSH logins though. > > A few days went by, and we got a phone call from ComCast about a report they > got from a company (whom they wouldn't name) basically saying that they were > being FTP attacked from our IP address. So we pulled the server off line, > ran a rootkit scan which came back clean, did a little more tweaking, took a > look at our iptables config and our router set up (80, 443, 22, 125(3?), > 10000 were the ports we had forwarded) and everything seemed ok. So we put > it back online. The next day, we reviewed the router logs and there were > thousands of STORM ATTACK entries (we were being ddos'd?) and several unique > IP's that were foreign to us. We did a few lookups/traces... > > > pug@jabberwocky:~$ whois -H --verbose 79.115.177.0 > Using server whois.ripe.net. > Query string: "-V Md5.0 79.115.177.0" > > % This is the RIPE Database query service. > % The objects are in RPSL format. > % > % The RIPE Database is subject to Terms and Conditions. > % See http://www.ripe.net/db/support/db-terms-conditions.pdf > > % Note: this output has been filtered. > % To receive output for a database update, use the "-B" flag. > > % Information related to '79.115.132.0 - 79.115.187.255' > > inetnum: 79.115.132.0 - 79.115.187.255 > netname: RO-RCS-RDS-FIBERLINK > descr: RCS & RDS S.A. > descr: FiberLink Customers > descr: Bucuresti > country: RO > admin-c: RDS-RIPE > tech-c: RDS-RIPE > status: ASSIGNED PA > mnt-by: AS8708-MNT > source: RIPE # Filtered > > role: Romania Data Systems NOC > address: 71-75 Dr. Staicovici > address: Bucharest / ROMANIA > phone: +40 21 30 10 888 > fax-no: +40 21 30 10 892 > abuse-mailbox: [email protected] > admin-c: CN19-RIPE > admin-c: VIG10-RIPE > tech-c: CN19-RIPE > tech-c: VIG10-RIPE > nic-hdl: RDS-RIPE > mnt-by: AS8708-MNT > remarks: > +--------------------------------------------------------------+ > remarks: | ABUSE CONTACT: [email protected] IN CASE OF HACK ATTACKS, > | > remarks: | ILLEGAL ACTIVITY, VIOLATION, SCANS, PROBES, SPAM, ETC. > | > remarks: | !! PLEASE DO NOT CONTACT OTHER PERSONS FOR THESE PROBLEMS > !! | > remarks: > +--------------------------------------------------------------+ > source: RIPE # Filtered > > % Information related to '79.112.0.0/13AS8708' > > route: 79.112.0.0/13 > descr: RDSNET > origin: AS8708 > mnt-by: AS8708-MNT > source: RIPE # Filtered > > > > > > And our traceroute (from a different IP out of our routers logs)... > > slippy@Slippy:~$ traceroute 79.115.187.255 > traceroute to 79.115.187.255 (79.115.187.255), 30 hops max, 60 byte packets > 1 192.168.1.1 (192.168.1.1) 2.476 ms 7.208 ms 7.600 ms > 2 10.19.14.1 (10.19.14.1) 27.978 ms 29.492 ms 30.777 ms > 3 static-64-222-84-42.burl.east.myfairpoint.net (64.222.84.42) 32.944 ms > 34.313 ms 35.889 ms > 4 POS4-0-1.GW16.NYC9.ALTER.NET (208.192.176.205) 51.879 ms 53.312 ms > 54.770 ms > 5 0.ge-3-0-0.XT1.NYC9.ALTER.NET (152.63.22.138) 56.072 ms 57.433 ms > 58.874 ms > 6 0.so-6-0-2.XL3.NYC4.ALTER.NET (152.63.10.21) 60.336 ms 40.160 ms > 67.725 ms > 7 GigabitEthernet4-0-0.GW1.NYC4.ALTER.NET (152.63.20.97) 43.537 ms > GigabitEthernet6-0-0.GW1.NYC4.ALTER.NET (152.63.20.49) 45.893 ms > GigabitEthernet4-0-0.GW1.NYC4.ALTER.NET (152.63.20.97) 46.337 ms > 8 teliasonera-gw.customer.alter.net (157.130.60.14) 49.415 ms 49.822 ms > 53.146 ms > 9 nyk-bb2-link.telia.net (213.155.130.244) 68.458 ms 70.546 ms 70.952 ms > 10 kbn-bb2-link.telia.net (80.91.247.120) 226.735 ms 239.932 ms 240.841 ms > 11 hbg-bb2-link.telia.net (80.91.249.207) 168.734 ms 170.594 ms 171.583 ms > 12 win-bb2-link.telia.net (80.91.246.25) 149.003 ms win-bb2-link.telia.net > (213.155.131.111) 151.061 ms win-bb2-link.telia.net (80.91.246.25) 154.174 > ms > 13 buca-b1-link.telia.net (80.91.246.155) 192.605 ms 193.127 ms > buca-b1-link.telia.net (80.91.247.81) 174.068 ms > 14 rcsrds-ic-141235-buca-b1.c.telia.net (213.248.83.170) 176.061 ms > 176.270 ms 177.728 ms > 15 213-154-124-95.rdsnet.ro (213.154.124.95) 185.042 ms 186.032 ms > 186.439 ms > > Our Routers logs have thousands of entries akin to the following > [DoS attack: STORM] attack packets in last 20 sec from ip [213.161.192.51], > Tuesday, May 03,2011 11:29:17 > > [DoS attack: Smurf] attack packets in last 20 sec from ip [82.176.5.255], > Tuesday, May 03,2011 11:28:59 > > > Naturally, we emailed their ISP reporting the incident - but it's romania, > who cares? > At any rate, this is where we began to scratch our heads. A day or so later, > I was logged in remotely and just to see if I could see Dave logged in, I ran > a > pug@jabberwocky:~$ who -a > 2011-05-04 09:12 415 id=si term=0 > exit=0 > system boot 2011-05-04 09:12 > run-level 2 2011-05-04 09:12 last=S > 2011-05-04 09:12 1048 id=l2 term=0 > exit=0 > LOGIN tty1 2011-05-04 09:12 1876 id=1 > LOGIN tty6 2011-05-04 09:12 1881 id=6 > LOGIN tty3 2011-05-04 09:12 1878 id=3 > LOGIN tty4 2011-05-04 09:12 1879 id=4 > LOGIN tty5 2011-05-04 09:12 1880 id=5 > LOGIN tty2 2011-05-04 09:12 1877 id=2 > nagios + pts/0 2011-05-04 15:19 00:06 14217 (79.115.177.0) > pug + pts/1 2011-05-04 12:40 00:16 7866 > (pool-72-92-154-71.burl.east.myfairpoint.net) > pug + pts/2 2011-05-04 15:29 . 15633 > (pool-72-92-154-71.burl.east.myfairpoint.net) > pug@jabberwocky:~$ > > and saw our local nagios user logged in from that IP address. So naturally, > we kicked him out. > > After that, we KNEW we had unauthorized remote access so we took the server > down again. At this point, we called up our local comcast techie (the guy > that called us to inform us what was happening) and he said that he had the > same issue with his home server. He wound up banning the African, Asian and > South American continents with iptables - which is something that I would > like to do. > > But this left us with questions. > > 1) How did they gain access? I doubt that it would be possible for them to > bruteforce that password - it was 10 characters long, upper and lower case, > numbers and symbols. So then what vulnerability did they exploit to get in? > > 2) What else do I need to do to secure a Debian install for dedicated online > time plugged into the ebil interwebs? > > We now are starting over, on both servers. We have fresh Debian installs (no > gui - why bother?). The list of things to do that I have come up with is as > follows > > Install Linux > Install Snort-mysql and acidbase w/ addons & configure with/for IPTables > Install Nagios (is there some vulnerability to this?) > Install Webmin > Install Cactus > Ensure that SSH is listening/accepting logins with keys ONLY (it is, now) > This is all before we do anything else - our LAMP stack will be installed at > the end anyway having installed everything with snort. Is there anything > else that we should be taking in to consideration / that we should do? And, > having played around with a snort install... I am WAY over my head, I have no > idea how the classes of IP blocks break down, or how to set SNORT to listen > on a single adapter (as the router only forwards ports to the server, nowhere > else on the network). > > I'm looking forward to the next meeting as I am thinking about showing up! :)
