FW: FW: monitor ALL connections to ALL ports
-Original Message- From: Peter Pentchev [mailto:[EMAIL PROTECTED]] Sent: 16 October 2002 11:37 To: Carroll, D. (Danny) Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: FW: monitor ALL connections to ALL ports On Wed, Oct 16, 2002 at 10:48:01AM +0200, [EMAIL PROTECTED] wrote: Something else you could do, if you want to put the effort into it is to write a program that accepts all packets from ipfw (via a divert rule) and then logs what you want before returning the untouched packed back to ipfw. Much like what natd does, except without the natting. I am sure the natd sources would be very useful in this case. I am a bit surprised that nobody has mentioned ports/net/clog yet. It is simple yet effective; it does not log UDP packets, but this functionality may not be too hard to add. G'luck, Peter -- Peter Pentchev [EMAIL PROTECTED][EMAIL PROTECTED] PGP key:http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 This sentence every third, but it still comprehensible. - ATTENTION: The information in this electronic mail message is private and confidential, and only intended for the addressee. Should you receive this message by mistake, you are hereby notified that any disclosure, reproduction, distribution or use of this message is strictly prohibited. Please inform the sender by reply transmission and delete the message without copying or opening it. Messages and attachments are scanned for all viruses known. If this message contains password-protected attachments, the files have NOT been scanned for viruses by the ING mail domain. Always scan attachments before opening them. - ATT6.dat Description: PGP signature
Sizing /var (was Re: monitor ALL connections to ALL ports)
On Tue, Oct 15, 2002 at 10:35:21AM +0800, Roman V. Mashak wrote: Could you describe some methods of counting /var-partition size for saving there: 1) 'maillog' data 2) 'ipfw' logs. That's a very hard question to answer. It depends on so many different variables --- how much traffic you're seeing, how much you choose to log, how long you keep log files around and even how compressible the resulting log files are. Now, there's no point getting too precise with all this. On machines I set up for personal use I tend to create a /var partition of 128Mb, and on my home machine at the moment /var is running at 35% percent full. That's fine --- plenty of room for growth or unexpected peaks in traffic (remember the nimda worm?). I wouldn't be too unhappy if usage had stabilized at anywhere up to about 75% full. For most uses a 128Mb /var partition should be fine. The exceptions are: i) You may need plenty of space in /var/crash if you're going to be debugging system crashes. Each crashdump will require slightly more space than the total RAM in your machine, and you usually need to have several sets to work with. Nb. /var/crash is the traditional place to store crash dumps, but it's easy enough to configure the system to use a different partition, which is what I do. ii) You run a particularly busy server --- say your mail or web server gets 100,000 hits in a day and each hit results in about 200 bytes of log message. That's approximately 20Mb a day. Without compression, that's enough to fill up a 128Mb partition inside a week. Assuming you get 80% compression with gzip (not unreasonable for log files) that will give you space for roughly a month's worth of log files. iii) You log an unreasonably large amount of stuff. Suppose the average size of web page (or mail message) on your server is 15kb. You choose to log every http / smtp packet your server deals with --- with a MTU of 1500 bytes that's 10 packets just for sending out the web page or message. So we're looking at approximately a 10 fold increase in the amount of logging data to deal with over (ii), or enough to overflow a 128Mb partition in less than a day without compression. Now, those numbers are approximate, but not unrealistic. I've taken no account of all the other stuff that lives in /var, but that tends to be reasonably constant in size. The best way to proceed is to make this sort of rough calculation to get a ball-park idea of what the right size should be, add some extra for luck and then try it out. Keep a record of how much of the partition is in use each day and examine the trends to see whether it's going to stabilize at around a reasonable percentage. If not, then you can fiddle with the settings in /etc/newsyslog.conf or switch to bzip2 compression or (if the worst comes to the worst) mount a larger partition on /var/log and next time you have the machine scheduled for major maintenance rebuild it with a bigger /var. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks Savill Way Marlow Tel: +44 1628 476614 Bucks., SL7 1TH UK To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
RE: monitor ALL connections to ALL ports
This is what I currently have. /dev/ad0s3e 1.2G 912M 175M84%/var/log I got a 30 count of sys rotates at ever 10 megs (newsyslog.conf), expect for httpd* logs, I just leave them untouched for stat info and clean by hand, right now htye are only 115 megs. Jack -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Roman V. Mashak Sent: Monday, October 14, 2002 9:35 PM To: [EMAIL PROTECTED] Subject: Re: monitor ALL connections to ALL ports On Mon, Oct 14, 2002 at 11:42:25PM +0100, Matthew Seaman wrote: I want to log all connections, regardless if they failed or successed, regardless if they have a daemon running on that port or not. The only way I can think of to achieve what you want -- logging every packet received by your machine -- is to use ipfw(8) and add the 'log' keyword to all appropriate rules. You'll need to have a lot of space in /var and bump up the net.inet.ip.fw.verbose_limit sysctl to some Could you describe some methods of counting /var-partition size for saving there: 1) 'maillog' data 2) 'ipfw' logs. Thanks in advance. huge limit and run 'ipfw resetlog' at regular intervals (or ipfw(8) [skip] -- Best regards, Roman To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: monitor ALL connections to ALL ports
It sounds to me that you are looking for a Network Intrusion system. 1) try: /usr/ports/security/snort It has plenty of rules that can help you log whatever u like. 2) Also another possibility is to use tcpdump host YOUR_IP -w LOGFILE which will log all the packets heading for your IP in raw form in the logifle. TCPdump has many swithches. The format of the logfile is in libcap format and there are plenty of parsers of this file [including tcpdump, ethereal, snort] IMO, try to log ALL connections to ALL ports ONLY if ur box is faster than a PIII, 500Mhz, 256 RAM. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
RE: monitor ALL connections to ALL ports
I moved this thread to [EMAIL PROTECTED] due to a request that [EMAIL PROTECTED] was an inapropate place to discuss this issuse. On Mon, Oct 14, 2002 at 02:31:05PM -0500, Maildrop wrote: I put these rule in: ipfw add count log all from any to any I am getting messages in my log (/var/log/all.log) that appears like this: Oct 14 14:15:06 hydra /kernel: Connection attempt to UDP 192.168.17.1:161 from 192.168.17.1:1166 That's a log_in_vain message, not ipfw(8). Which is exactly, what I want, but there is a couple isses: 1) It only logs failed connects. If I try to `telnet localhost 55`, it will log that, but if I do a `telnet locahost 80` (where web server is running) the connection is valid and doesn't log it. Right, that's how log_in_vain works. (from tcp(4)) tcp.log_in_vainLog any connection attempts to ports where there is not a socket accepting connections. The value of 1 limits the logging to SYN (connection establishment) packets only. That of 2 results in any TCP packets to closed ports being logged. Any value unlisted above disables the logging (default is 0, i.e., the logging is disabled). '1' is limited to connection established (valid connections) and '2' is limited to connection failed... how do I get both failed AND established from log_in_vain? I want to log all connections, regardless if they failed or successed, regardless if they have a daemon running on that port or not. Currently, they are both set as '1': net.inet.tcp.log_in_vain: 1 net.inet.udp.log_in_vain: 1 2) How do I setup Syslog for this? ipfw man page says it logs to LOG_SECURITY facility. I want to log all connections (failed or not), into one file.. This is what I currently have in my syslogd.conf file (the log above I am pulling from all.log): security.* /var/log/security log.security/var/log/ipfw.log Both these files are empty :( I restarted syslogd. The second one should give you an error. The first one should catch ipfw(8) logging. You did rebuild your kernel with IPFIREWALL and IPFIREWALL_VERBOSE, right? Yep. 4.7-release: options IPFIREWALL options IPDIVERT options IPFIREWALL_VERBOSE #enable logging to syslogd(8) options IPFIREWALL_VERBOSE_LIMIT=100#limit verbosity options IPSTEALTH #support for stealth forwarding Kernel that is currently running (from kernel config above, clean reboot and didn't change anything sysctl): net.inet.ip.fw.enable: 1 net.inet.ip.fw.one_pass: 1 net.inet.ip.fw.debug: 1 net.inet.ip.fw.verbose: 1 net.inet.ip.fw.verbose_limit: 100 net.inet.ip.fw.dyn_buckets: 256 net.inet.ip.fw.curr_dyn_buckets: 256 net.inet.ip.fw.dyn_count: 0 net.inet.ip.fw.dyn_max: 1000 net.inet.ip.fw.static_count: 7 net.inet.ip.fw.dyn_ack_lifetime: 300 net.inet.ip.fw.dyn_syn_lifetime: 20 net.inet.ip.fw.dyn_fin_lifetime: 1 net.inet.ip.fw.dyn_rst_lifetime: 1 net.inet.ip.fw.dyn_udp_lifetime: 10 net.inet.ip.fw.dyn_short_lifetime: 5 net.inet.ip.fw.dyn_grace_time: 10 net.link.ether.ipfw: 0 Something weird that I found: hydra# ipmon -D /var/log/ipfw.log /dev/ipl: open: Device not configured hydra# file /dev/ipl /dev/ipl: character special (79/0) hydra# grep ipmon /etc/rc.conf ipmon_enable=NO # Set to YES for ipmon; needs ipfilter or ipnat ipmon_program=/sbin/ipmon # where the ipfilter monitor program lives ipmon_flags=-Ds # typically -Ds or -D /var/log/ipflog Is ipmon part of ipfw? hydra# cd /var/log hydra# ls -l ipfw* -rw-r--r-- 1 root wheel0 Oct 14 13:26 ipfw.log -rw--- 1 root wheel 163 Oct 13 03:05 ipfw.today -rw--- 1 root wheel 151 Oct 12 03:07 ipfw.yesterday hydra# cat ipfw.log hydra# cat ipfw.today 00200 0 0 deny ip from any to 127.0.0.0/8 00300 0 0 deny ip from 127.0.0.0/8 to any 65535 0 0 deny ip from any to any hydra# cat ipfw.yesterday 00200 00 deny ip from any to 127.0.0.0/8 00300 00 deny ip from 127.0.0.0/8 to any 65535 00 deny ip from any to any hydra# ipfw list 00050 divert 8668 ip from any to any via dc1 00100 allow ip from any to any via lo0 00200 deny ip from any to 127.0.0.0/8 00300 deny ip from 127.0.0.0/8 to any 65000 allow ip from any to any 65100 count log logamount 100 ip from any to any 65535 deny ip from any to any To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: monitor ALL connections to ALL ports
On Mon, Oct 14, 2002 at 05:09:43PM -0500, Maildrop wrote: On Mon, Oct 14, 2002 at 02:31:05PM -0500, Maildrop wrote: I put these rule in: ipfw add count log all from any to any I am getting messages in my log (/var/log/all.log) that appears like this: Oct 14 14:15:06 hydra /kernel: Connection attempt to UDP 192.168.17.1:161 from 192.168.17.1:1166 That's a log_in_vain message, not ipfw(8). Which is exactly, what I want, but there is a couple isses: 1) It only logs failed connects. If I try to `telnet localhost 55`, it will log that, but if I do a `telnet locahost 80` (where web server is running) the connection is valid and doesn't log it. Right, that's how log_in_vain works. (from tcp(4)) tcp.log_in_vainLog any connection attempts to ports where there is not a socket accepting connections. The value of 1 limits the logging to SYN (connection establishment) packets only. That of 2 results in any TCP packets to closed ports being logged. Any value unlisted above disables the logging (default is 0, i.e., the logging is disabled). '1' is limited to connection established (valid connections) No, it's limited to SYN's at _closed_ ports. Read the first sentence again, Log any connection attempts to ports where there is not a socket accepting connections. and '2' is limited to connection failed... how do I get both failed AND established from log_in_vain? I want to log all connections, regardless if they failed or successed, regardless if they have a daemon running on that port or not. log_in_vain doesn't do connections to listening ports. That's the job of what ever is listening. Currently, they are both set as '1': net.inet.tcp.log_in_vain: 1 net.inet.udp.log_in_vain: 1 2) How do I setup Syslog for this? ipfw man page says it logs to LOG_SECURITY facility. I want to log all connections (failed or not), into one file.. This is what I currently have in my syslogd.conf file (the log above I am pulling from all.log): security.* /var/log/security log.security/var/log/ipfw.log Both these files are empty :( I restarted syslogd. The second one should give you an error. The first one should catch ipfw(8) logging. You did rebuild your kernel with IPFIREWALL and IPFIREWALL_VERBOSE, right? [snip] Something weird that I found: hydra# ipmon -D /var/log/ipfw.log /dev/ipl: open: Device not configured hydra# file /dev/ipl /dev/ipl: character special (79/0) hydra# grep ipmon /etc/rc.conf ipmon_enable=NO # Set to YES for ipmon; needs ipfilter or ipnat ipmon_program=/sbin/ipmon # where the ipfilter monitor program lives ipmon_flags=-Ds # typically -Ds or -D /var/log/ipflog Is ipmon part of ipfw? No, it's part of IPFilter. Here's your problem, hydra# ipfw list 00050 divert 8668 ip from any to any via dc1 00100 allow ip from any to any via lo0 00200 deny ip from any to 127.0.0.0/8 00300 deny ip from 127.0.0.0/8 to any 65000 allow ip from any to any 65100 count log logamount 100 ip from any to any 65535 deny ip from any to any How is anything ever going to reach rule 65100? 65000 passes everything. -- Crist J. Clark | [EMAIL PROTECTED] | [EMAIL PROTECTED] http://people.freebsd.org/~cjc/| [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: monitor ALL connections to ALL ports
On Mon, Oct 14, 2002 at 05:09:43PM -0500, Maildrop wrote: I want to log all connections, regardless if they failed or successed, regardless if they have a daemon running on that port or not. The only way I can think of to achieve what you want -- logging every packet received by your machine -- is to use ipfw(8) and add the 'log' keyword to all appropriate rules. You'll need to have a lot of space in /var and bump up the net.inet.ip.fw.verbose_limit sysctl to some huge limit and run 'ipfw resetlog' at regular intervals (or ipfw(8) will quit logging packets --- that's a measure introduced to prevent the blackhats DoS'ing a machine by causing so many log messages to be generated it fills up the disk). You understand that if you make any significant use of networking on your machine, configuring ipfw(8) in that way will result in you being drowned in such a flood of log messages you probably won't be able to cope. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks Savill Way Marlow Tel: +44 1628 476614 Bucks., SL7 1TH UK To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: monitor ALL connections to ALL ports
On Mon, Oct 14, 2002 at 11:42:25PM +0100, Matthew Seaman wrote: I want to log all connections, regardless if they failed or successed, regardless if they have a daemon running on that port or not. The only way I can think of to achieve what you want -- logging every packet received by your machine -- is to use ipfw(8) and add the 'log' keyword to all appropriate rules. You'll need to have a lot of space in /var and bump up the net.inet.ip.fw.verbose_limit sysctl to some Could you describe some methods of counting /var-partition size for saving there: 1) 'maillog' data 2) 'ipfw' logs. Thanks in advance. huge limit and run 'ipfw resetlog' at regular intervals (or ipfw(8) [skip] -- Best regards, Roman To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
