Re: How can I get all IP transactions (in/out) logged?
Yes iptables can do this. I know iptables can log to syslog, and believe there are ways to make it log to SQL, but I am unfamiliar with those. . . ... Thanks - that sounds like a plan... (I knew I wouldn't escape dealing with iptables). I would think this capability would be built into one of the firewall products, but I haven't found it. There may be a configuration setting to get iptables to log to something other than syslog, also (I know it's possible with pppd, although the logs seem to go to both places rather than just one). I can postprocess the log file to reduce the data... Thanks again - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How can I get all IP transactions (in/out) logged?
On Wed, Jul 28, 2004 at 10:39:07AM +0800, John Summerfield wrote: | James Foster wrote: | | The log will be _enormous_ and I mean __enormous__ | | It seems to me that the log won't necessarily be very large. It really | depends on how the connection is being used, doesn't it? An hours | worth of log from a dialup connection couldn't be very large, for | example. | | | I regularly pull 5K bytes/sec. That's a lot of transactions p/h, even on | dialup. Bandwidth does not (directly) affect the number of transactions. Just imagine a trivial program designed to DoS the requested logger. All it has to do is open and close a connection repeatedly. That requires sending a SYN, receiving a SYN-ACK, sending an ACK, then sending the FIN sequence. This is only a handful of bytes, so the bandwidth limitations of a dial-up connection won't prevent the logs from growing very large very rapidly. -D -- Love is not affectionate feeling, but a steady wish for the loved person's ultimate good as far as it can be obtained. --C.S. Lewis www: http://dman13.dyndns.org/~dman/jabber: [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: How can I get all IP transactions (in/out) logged?
It seems to me that the log won't necessarily be very large. It really depends on how the connection is being used, doesn't it? An hours worth of log from a dialup connection couldn't be very large, for example. Of course, on a broadband connection with lots of websites being visited or files being downloaded, the log would become quite large fairly quickly. It would depend on how much information is logged. Logging the contents of packets during a web surfing session would generate a large file. But, all I'm interested in is a source and destination IP for what has gone in and out of my system, along with possibly what port was used, what process ID was using the port, a timestamp, and a packet count. I found a reference on the linuxsecurity website to some sort of utility that will troll the various logfiles in an attempt to reconstruct some of this information, ostensibly in the aftermath of a successful cracking attempt. Another reference suggests running a packet sniffer (snort?) on your system. It seems rather silly to have to resort to that sort of thing, when enough system access is available to facilitate tracking network activity proactively. But what do I know... even my experience at breaking and entering is sadly outdated... -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How can I get all IP transactions (in/out) logged?
Yes iptables can do this. I know iptables can log to syslog, and believe there are ways to make it log to SQL, but I am unfamiliar with those. With syslog logging, you will also probably want to look into a syslog replacement such as syslog-ng where you can separate log data to multiple files much easier than the usual channels plain syslog provides. To do this you use the target -j LOG with one or two additional options: iptables -A input -i ppp0 -j LOG --log-level debug --log-prefix PACKETLOG-IN iptables -A output -o ppp0 -j LOG --log-level debug --log-prefix PACKETLOG-OUT iptables -A forward -i ppp0 -j LOG --log-level debug --log-prefix PACKETLOG-IN iptables -A forward -o ppp0 -j LOG --log-level debug --log-prefix PACKETLOG-OUT You can use whichever combination of the above four rules depending on what you want to log. #1 logs packets in ppp0 destined to the router/host machine #2 logs packets out ppp0 sourced from the router/host machine #3 logs packets in ppp0 destined to something else on your network #4 logs packets out ppp0 sourced from something else on your network --log-level has to be a valid syslog type (man syslog.conf) --log-prefix is a 14 char string prepended to the log message. This can be changed to suit your needs, and I would suggest configuring syslog-ng to pattern match for this string so you can store each iptables rule's output in a separate file as needed. You can have two files, in and out, each mixed no matter what its to/from (as in my examples above) or have the forward and 'to router host' separated, or any combinations you need. These logs will get large the more data you pass. If you store them for say 24 hours and rotate them as you pass them off to some other accounting/processing software (one that just stores summary info for example) and bzip2's the logs, deleting after a couple days, this shouldn't be bad on a dialup. -- Jon On Tue, 27 Jul 2004 12:27:56 -0700, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: I just want a basic log file containing the source and destination addresses for all traffic in and out of via PPP, so that I can keep track of what connections to outside IP addresses are made, and from where (externally, or from something running on my system) they originated. I've turned on the debug mode in PPP but it doesn't seem to provide what I want. For instance, it logs transactions from my system to my ISP, but doesn't log what's happening with any greater resolution (for instance, if I ping a system on the Internet, there's no record of the ping attempt in any of the system log files). The firewall as set up by firestarter logs blocked firewall penetration attempts, but not legitimate transactions in and out (perhaps there's a method, which I haven't found, to change that?) Is there any built-in facility that will accomplish this, or is it necessary/possible to construct something using iptables, or is there a contributed app that will do it? I realize that there are log files for the various servers and utilities (FTP, etc.) that contain this information on a piecemeal basis, but it seems to me that it would make more sense to collect it at the connection point(s) in and out of the system, at least on an interface-by-interface basis. Once again, I'm surprised that a Web search and various site searches haven't turned up a solution to this. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
RE: How can I get all IP transactions (in/out) logged?
I am pretty sure freshmeat.net has a tool that uses iptables and puts it into a mysql database...try looking under monitor or uptime its there somewhere. regards S -Original Message- From: Jon [mailto:[EMAIL PROTECTED] Sent: Thursday, 29 July 2004 10:17 a.m. To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: How can I get all IP transactions (in/out) logged? Yes iptables can do this. I know iptables can log to syslog, and believe there are ways to make it log to SQL, but I am unfamiliar with those. With syslog logging, you will also probably want to look into a syslog replacement such as syslog-ng where you can separate log data to multiple files much easier than the usual channels plain syslog provides. To do this you use the target -j LOG with one or two additional options: iptables -A input -i ppp0 -j LOG --log-level debug --log-prefix PACKETLOG-IN iptables -A output -o ppp0 -j LOG --log-level debug --log-prefix PACKETLOG-OUT iptables -A forward -i ppp0 -j LOG --log-level debug --log-prefix PACKETLOG-IN iptables -A forward -o ppp0 -j LOG --log-level debug --log-prefix PACKETLOG-OUT You can use whichever combination of the above four rules depending on what you want to log. #1 logs packets in ppp0 destined to the router/host machine #2 logs packets out ppp0 sourced from the router/host machine #3 logs packets in ppp0 destined to something else on your network #4 logs packets out ppp0 sourced from something else on your network --log-level has to be a valid syslog type (man syslog.conf) --log-prefix is a 14 char string prepended to the log message. This can be changed to suit your needs, and I would suggest configuring syslog-ng to pattern match for this string so you can store each iptables rule's output in a separate file as needed. You can have two files, in and out, each mixed no matter what its to/from (as in my examples above) or have the forward and 'to router host' separated, or any combinations you need. These logs will get large the more data you pass. If you store them for say 24 hours and rotate them as you pass them off to some other accounting/processing software (one that just stores summary info for example) and bzip2's the logs, deleting after a couple days, this shouldn't be bad on a dialup. -- Jon On Tue, 27 Jul 2004 12:27:56 -0700, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: I just want a basic log file containing the source and destination addresses for all traffic in and out of via PPP, so that I can keep track of what connections to outside IP addresses are made, and from where (externally, or from something running on my system) they originated. I've turned on the debug mode in PPP but it doesn't seem to provide what I want. For instance, it logs transactions from my system to my ISP, but doesn't log what's happening with any greater resolution (for instance, if I ping a system on the Internet, there's no record of the ping attempt in any of the system log files). The firewall as set up by firestarter logs blocked firewall penetration attempts, but not legitimate transactions in and out (perhaps there's a method, which I haven't found, to change that?) Is there any built-in facility that will accomplish this, or is it necessary/possible to construct something using iptables, or is there a contributed app that will do it? I realize that there are log files for the various servers and utilities (FTP, etc.) that contain this information on a piecemeal basis, but it seems to me that it would make more sense to collect it at the connection point(s) in and out of the system, at least on an interface-by-interface basis. Once again, I'm surprised that a Web search and various site searches haven't turned up a solution to this. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How can I get all IP transactions (in/out) logged?
[EMAIL PROTECTED] wrote: It seems to me that the log won't necessarily be very large. It really depends on how the connection is being used, doesn't it? An hours worth of log from a dialup connection couldn't be very large, for example. Of course, on a broadband connection with lots of websites being visited or files being downloaded, the log would become quite large fairly quickly. It would depend on how much information is logged. Logging the contents of packets during a web surfing session would generate a large file. But, all I'm interested in is a source and destination IP for what has gone in and out of my system, along with possibly what port was used, what process ID was using the port, a timestamp, and a packet count. The kernel can log stuff like this: Shorewall:net2all:DROP:IN=ppp0 OUT= MAC= SRC=219.150.118.21 DST=202.89.174.209 LEN=1106 TOS=0x00 PREC=0x00 TTL=101 ID=28911 PROTO=UDP SPT=3790 DPT=1026 LEN=1086 Still, you're likely to get big logs. For more info man iptables http://www.netfilter.org/ -- Cheers John -- spambait [EMAIL PROTECTED] [EMAIL PROTECTED] Tourist pics http://portgeographe.environmentaldisasters.cds.merseine.nu/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
How can I get all IP transactions (in/out) logged?
I just want a basic log file containing the source and destination addresses for all traffic in and out of via PPP, so that I can keep track of what connections to outside IP addresses are made, and from where (externally, or from something running on my system) they originated. I've turned on the debug mode in PPP but it doesn't seem to provide what I want. For instance, it logs transactions from my system to my ISP, but doesn't log what's happening with any greater resolution (for instance, if I ping a system on the Internet, there's no record of the ping attempt in any of the system log files). The firewall as set up by firestarter logs blocked firewall penetration attempts, but not legitimate transactions in and out (perhaps there's a method, which I haven't found, to change that?) Is there any built-in facility that will accomplish this, or is it necessary/possible to construct something using iptables, or is there a contributed app that will do it? I realize that there are log files for the various servers and utilities (FTP, etc.) that contain this information on a piecemeal basis, but it seems to me that it would make more sense to collect it at the connection point(s) in and out of the system, at least on an interface-by-interface basis. Once again, I'm surprised that a Web search and various site searches haven't turned up a solution to this. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How can I get all IP transactions (in/out) logged?
[EMAIL PROTECTED] wrote: I just want a basic log file containing the source and destination addresses for all traffic in and out of via PPP, so that I can keep track of what connections to outside IP addresses are made, and from where (externally, or from something running on my system) they originated. I've turned on the debug mode in PPP but it doesn't seem to provide what I want. For instance, it logs transactions from my system to my ISP, but doesn't log what's happening with any greater resolution (for instance, if I ping a system on the Internet, there's no record of the ping attempt in any of the system log files). The firewall as set up by firestarter logs blocked firewall penetration attempts, but not legitimate transactions in and out (perhaps there's a method, which I haven't found, to change that?) Is there any built-in facility that will accomplish this, or is it necessary/possible to construct something using iptables, or is there a contributed app that will do it? Isn't iptables built in? The log will be _enormous_ and I mean __enormous__ I realize that there are log files for the various servers and utilities (FTP, etc.) that contain this information on a piecemeal basis, but it seems to me that it would make more sense to collect it at the connection point(s) in and out of the system, at least on an interface-by-interface basis. Once again, I'm surprised that a Web search and various site searches haven't turned up a solution to this. Probably because most people think this a fine way to hang themselves:-) -- Cheers John -- spambait [EMAIL PROTECTED] [EMAIL PROTECTED] Tourist pics http://portgeographe.environmentaldisasters.cds.merseine.nu/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How can I get all IP transactions (in/out) logged?
The log will be _enormous_ and I mean __enormous__ It seems to me that the log won't necessarily be very large. It really depends on how the connection is being used, doesn't it? An hours worth of log from a dialup connection couldn't be very large, for example. Of course, on a broadband connection with lots of websites being visited or files being downloaded, the log would become quite large fairly quickly. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
RE: How can I get all IP transactions (in/out) logged?
You could run a squid proxy on the box and redirect http requests, this will log. Assuming you only want http of course. I just looked at my squid logs, for 1 squid server (we run 2 in parallel with DNS round-robin) the log is 350meg to 1.3gig per day per serverthats just http regards S -Original Message- From: James Foster [mailto:[EMAIL PROTECTED] Sent: Wednesday, 28 July 2004 1:54 p.m. To: debian helping Subject: Re: How can I get all IP transactions (in/out) logged? The log will be _enormous_ and I mean __enormous__ It seems to me that the log won't necessarily be very large. It really depends on how the connection is being used, doesn't it? An hours worth of log from a dialup connection couldn't be very large, for example. Of course, on a broadband connection with lots of websites being visited or files being downloaded, the log would become quite large fairly quickly. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: How can I get all IP transactions (in/out) logged?
James Foster wrote: The log will be _enormous_ and I mean __enormous__ It seems to me that the log won't necessarily be very large. It really depends on how the connection is being used, doesn't it? An hours worth of log from a dialup connection couldn't be very large, for example. I regularly pull 5K bytes/sec. That's a lot of transactions p/h, even on dialup. It does depend on _how much_ is logged. -- Cheers John -- spambait [EMAIL PROTECTED] [EMAIL PROTECTED] Tourist pics http://portgeographe.environmentaldisasters.cds.merseine.nu/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
