Looks great, thanks so much! ./netsniff-ng -i eth0 -o /tmp/test/ -s --prefix snort.log. --interval 1MiB --user 1 --group 1 netsniff-ng 0.5.8-rc0 Running! Hang up with ^C!
^C Cannot set NIC flags! ls -alh /tmp/test total 9.0M drwxr-xr-x 2 daemon daemon 4.0K Jan 22 06:28 . drwxrwxrwt 13 root root 12K Jan 22 06:25 .. -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:26 snort.log.1358854003.pcap -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:27 snort.log.1358854011.pcap -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:27 snort.log.1358854026.pcap -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:27 snort.log.1358854044.pcap -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:28 snort.log.1358854064.pcap -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:28 snort.log.1358854085.pcap -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:28 snort.log.1358854090.pcap -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:28 snort.log.1358854091.pcap -rw-r--r-- 1 daemon daemon 448K Jan 22 06:28 snort.log.1358854113.pcap Looking forward to 0.5.8! Thanks, Doug On Mon, Jan 21, 2013 at 4:12 PM, Daniel Borkmann <[email protected]> wrote: > On Mon, Jan 21, 2013 at 5:50 PM, Doug Burks <[email protected]> wrote: >> Excellent, thanks! > > Should be fixed now. > >> On Mon, Jan 21, 2013 at 11:48 AM, Daniel Borkmann <[email protected]> >> wrote: >>> On 01/21/2013 05:46 PM, Doug Burks wrote: >>>> >>>> After removing all set_ioprio_*() calls in the files pcap_mmap.c, >>>> pcap_rw.c, pcap_sg.c, it's now rotating properly: >>>> >>>> ./netsniff-ng -i eth0 -o /tmp/test/ -s --prefix snort.log. --interval >>>> 1MiB --user 1 --group 1 >>>> netsniff-ng 0.5.8-rc0 >>>> Running! Hang up with ^C! >>>> >>>> ^C >>>> >>>> Cannot set NIC flags! >>>> >>>> >>>> ls -alh /tmp/test >>>> total 5.0M >>>> drwxr-xr-x 2 daemon daemon 4.0K Jan 21 11:44 . >>>> drwxrwxrwt 13 root root 12K Jan 21 11:41 .. >>>> -rw-r--r-- 1 root root 1.1M Jan 21 11:44 snort.log.1358786624.pcap >>>> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 snort.log.1358786641.pcap >>>> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 snort.log.1358786656.pcap >>>> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 snort.log.1358786674.pcap >>>> -rw-r--r-- 1 daemon daemon 683K Jan 21 11:45 snort.log.1358786691.pcap >>> >>> >>> Okay, so I will fix things up tonight and let you know when it's upstream. >>> >>> >>>> On Mon, Jan 21, 2013 at 11:36 AM, Daniel Borkmann <[email protected]> >>>> wrote: >>>>> >>>>> On 01/21/2013 05:28 PM, Daniel Borkmann wrote: >>>>>> >>>>>> >>>>>> On 01/21/2013 05:16 PM, Doug Burks wrote: >>>>>>> >>>>>>> >>>>>>> Hi Daniel, >>>>>>> >>>>>>> I tested using options similar to what we normally use. It writes the >>>>>>> first pcap correctly (although I notice that the ownership is >>>>>>> root:root instead of daemon:daemon), but once the 1MiB interval has >>>>>>> been reached and netsniff-ng tries to rotate to the second pcap, it >>>>>>> fails when setting io prio. >>>>>>> >>>>>>> ./netsniff-ng -i eth0 -o /tmp/test/ -s --prefix snort.log. --interval >>>>>>> 1MiB --user 1 --group 1 >>>>>>> netsniff-ng 0.5.8-rc0 >>>>>>> Running! Hang up with ^C! >>>>>>> >>>>>>> Failed to set io prio for pid! >>>>>>> >>>>>>> >>>>>>> ls -alh /tmp/test/ >>>>>>> total 3.3M >>>>>>> drwxr-xr-x 2 daemon daemon 4.0K Jan 21 11:08 . >>>>>>> drwxrwxrwt 13 root root 12K Jan 21 11:06 .. >>>>>>> -rw-r--r-- 1 root root 1.1M Jan 21 11:05 snort.log.1358784314.pcap >>>>>>> -rw-r--r-- 1 daemon daemon 24 Jan 21 11:05 snort.log.1358784337.pcap >>>>>>> >>>>>>> >>>>>>> Any ideas? >>>>>> >>>>>> >>>>>> >>>>>> Thanks for your feedback / tests! >>>>>> >>>>>> Ideas why it fails ... yes. >>>>>> >>>>>> The privileges are currently dropped right before entering the main >>>>>> loop, >>>>>> so >>>>>> the first file is still opened as root. Basically, the issue is that in >>>>>> the >>>>>> background netsniff-ng does some more automatic tuning that requires >>>>>> privileges >>>>>> (like setting the disc I/O scheduling policy), instead of just opening a >>>>>> file >>>>>> and that's it. >>>>>> >>>>>> So in the end, it's only useful for a couple of scenarios (for trafgen >>>>>> in >>>>>> general >>>>>> it's useful, netsniff-ng: for replaying a pcap, for recording a single >>>>>> pcap, ...), >>>>>> but probably not for the use case you have. Thus, I'm afraid, sticking >>>>>> with the >>>>>> setcap workaround might be the best solution. >>>>>> >>>>>> I don't see a value in shortly regaining privileges and then dropping >>>>>> them >>>>>> again >>>>>> (if that's actually possible with effective/read IDs); probably, it >>>>>> might >>>>>> be as >>>>>> good as not dropping them at all. >>>>>> >>>>>> Opinions? >>>>> >>>>> >>>>> >>>>> If it concerns only the ioprio, we could add another pcap function that >>>>> gets >>>>> called >>>>> once before dropping the privs. >>>>> >>>>> Would it work for you, if you remove all set_ioprio_*() calls in the >>>>> files >>>>> pcap_mmap.c, >>>>> pcap_rw.c, pcap_sg.c ? >>>>> >>>>> If so, then we can do sth about it! ;-) >>>>> >>>>> Let me know. Thanks! >>>>> >>>>> >>>>>>> On Mon, Jan 21, 2013 at 5:01 AM, Daniel Borkmann <[email protected]> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 01/19/2013 02:54 PM, Doug Burks wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks so much! >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I'd be grateful if you can test it on your system. >>>>>>>> >>>>>>>> It should work for most cases. In netsniff-ng, I've seen that it exits >>>>>>>> non-gracefully >>>>>>>> when netsniff-ng is in promisc mode, since after destroying the >>>>>>>> mmap'ed >>>>>>>> rings it tries >>>>>>>> to put the card into non-promisc again, but has no rights for it >>>>>>>> (since >>>>>>>> we've dropped >>>>>>>> it). I don't think it makes much sense to regain privileged rights >>>>>>>> only >>>>>>>> for >>>>>>>> the >>>>>>>> cleanup work. >>>>>>>> >>>>>>>> >>>>>>>>> On Fri, Jan 18, 2013 at 3:37 PM, Daniel Borkmann >>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Dec 2, 2012 at 10:11 PM, Doug Burks <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Well, since you asked... :) >>>>>>>>>>> >>>>>>>>>>> I know I can do the following to allow netsniff-ng to be run as a >>>>>>>>>>> non-root user: >>>>>>>>>>> sudo setcap >>>>>>>>>>> cap_net_raw,cap_ipc_lock,cap_sys_admin,cap_net_admin=eip >>>>>>>>>>> netsniff-ng >>>>>>>>>>> >>>>>>>>>>> But it would be quite nice if netsniff-ng had internal support for >>>>>>>>>>> dropping to a non-root user after opening the eth device. Many >>>>>>>>>>> other >>>>>>>>>>> pcap tools (such as daemonlogger, snort, suricata, etc.) support >>>>>>>>>>> this >>>>>>>>>>> via command-line arguments like: >>>>>>>>>>> -u user -g group >>>>>>>>>>> OR >>>>>>>>>>> --user user --group group >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Done, after setup, you can now drop privileges in netsniff-ng and >>>>>>>>>> trafgen. >>>>>>>>>> >>>>>>>>>>> Thoughts? >>>>>>>>>>> >>>>>>>>>>> Thanks for your consideration! >>>>>>>>>>> >>>>>>>>>>> Doug >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Dec 2, 2012 at 3:50 PM, Daniel Borkmann >>>>>>>>>>> <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> By the way, if you have any other feature requests / wishes >>>>>>>>>>>> (besides >>>>>>>>>>>> the list in TODO) that might be useful for many users, let us >>>>>>>>>>>> know, >>>>>>>>>>>> and we'd be happy to further improve the toolkit. >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Dec 2, 2012 at 5:49 PM, Daniel Borkmann >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Dec 2, 2012 at 5:47 PM, Doug Burks <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for the always fast response! Deploying the "tcpdump >>>>>>>>>>>>>> -dd" >>>>>>>>>>>>>> solution now. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> >>>>>>>>>>>>> If we have something new regarding bpfc, I'll announce it on the >>>>>>>>>>>>> list >>>>>>>>>>>>> anyway. >>>>>>>>>>>>> >>>>>>>>>>>>>> On Sun, Dec 2, 2012 at 9:00 AM, Daniel Borkmann >>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, Dec 2, 2012 at 2:55 PM, Doug Burks >>>>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, Oct 30, 2012 at 10:41 AM, Daniel Borkmann >>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>> <snip> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -f /etc/nsm/$HOSTNAME-$INTERFACE/bpf-pcap.conf >>>>>>>>>>>>>>>>>> This should be the same option in netsniff-ng, but my >>>>>>>>>>>>>>>>>> understanding is >>>>>>>>>>>>>>>>>> that I'll need to convert my "human-readable" bpf-pcap.conf >>>>>>>>>>>>>>>>>> using >>>>>>>>>>>>>>>>>> "tcpdump -dd"? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yes, it you want to use filters and bpf-pcap.conf contains >>>>>>>>>>>>>>>>> tcpdump-like filters, run them through "tcpdump -dd <my >>>>>>>>>>>>>>>>> filter>" > >>>>>>>>>>>>>>>>> out.ops and then pass out.ops to netsniff-ng via "--filter >>>>>>>>>>>>>>>>> out.ops". >>>>>>>>>>>>>>>>> That's it; netsniff-ng will then automatically enable the BPF >>>>>>>>>>>>>>>>> JIT >>>>>>>>>>>>>>>>> if >>>>>>>>>>>>>>>>> it's available in your kernel. This feature translates BPF >>>>>>>>>>>>>>>>> filters >>>>>>>>>>>>>>>>> into architecture optimized machine opcodes within the >>>>>>>>>>>>>>>>> kernel. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We've officially replaced daemonlogger with netsniff-ng and it >>>>>>>>>>>>>>>> appears >>>>>>>>>>>>>>>> to be working well! However, we haven't included BPF >>>>>>>>>>>>>>>> functionality >>>>>>>>>>>>>>>> yet, so I need to add that now. I can do what's described >>>>>>>>>>>>>>>> above, >>>>>>>>>>>>>>>> but >>>>>>>>>>>>>>>> the FAQ also says: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cool, I'm very happy about that! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> "If you try to create custom socket filters with tcpdump -dd, >>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>> to edit the ret opcode (0x6) of the resulting filter, >>>>>>>>>>>>>>>> otherwise >>>>>>>>>>>>>>>> your >>>>>>>>>>>>>>>> payload will be cut off: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 0x6, 0, 0, 0xFFFFFFFF instead of 0x6, 0, 0, 0x00000060 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The Linux kernel now takes skb->len instead of 0xFFFFFFFF. If >>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>> do >>>>>>>>>>>>>>>> not change it, the kernel will take 0x00000060 as buffer >>>>>>>>>>>>>>>> length >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>> packets larger than 96 Byte will be cut off (filled with zero >>>>>>>>>>>>>>>> Bytes)! >>>>>>>>>>>>>>>> It's a bug in libpcaps filter compiler. Detailed information >>>>>>>>>>>>>>>> about >>>>>>>>>>>>>>>> this issue can be found on our blog post." >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The linked blog post is no longer available. So is this an >>>>>>>>>>>>>>>> issue I >>>>>>>>>>>>>>>> need to be concerned about? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Actually not anymore. I use Fedora and the tcpdump version >>>>>>>>>>>>>>> there >>>>>>>>>>>>>>> outputs: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> # tcpdump -dd ip >>>>>>>>>>>>>>> tcpdump: WARNING: eth0: no IPv4 address assigned >>>>>>>>>>>>>>> { 0x28, 0, 0, 0x0000000c }, >>>>>>>>>>>>>>> { 0x15, 0, 1, 0x00000800 }, >>>>>>>>>>>>>>> { 0x6, 0, 0, 0x0000ffff }, >>>>>>>>>>>>>>> { 0x6, 0, 0, 0x00000000 }, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So they have changed this from 0x00000060 into 0x0000ffff. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> For bpfc itself, I didn't have time to finish the high-level >>>>>>>>>>>>>>> compiler, >>>>>>>>>>>>>>> yet. We have an assembler-like compiler where you can also >>>>>>>>>>>>>>> create >>>>>>>>>>>>>>> filters with, but for usability you can use the method >>>>>>>>>>>>>>> described >>>>>>>>>>>>>>> above, of course. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Doug Burks >>>>>>>>>>>>>> http://securityonion.blogspot.com >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Doug Burks >>>>>>>>>>> http://securityonion.blogspot.com >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> >>>> >>>> >>>> >>> >>> -- >>> >>> >> >> >> >> -- >> Doug Burks >> http://securityonion.blogspot.com >> >> -- >> >> > > -- > > -- Doug Burks http://securityonion.blogspot.com --
