I tested this and put in production. On Tue, Jan 22, 2013 at 6:51 AM, Daniel Borkmann <[email protected]> wrote: > On 01/22/2013 12:34 PM, Doug Burks wrote: >> >> Looks great, thanks so much! >> >> ./netsniff-ng -i eth0 -o /tmp/test/ -s --prefix snort.log. --interval >> 1MiB --user 1 --group 1 >> netsniff-ng 0.5.8-rc0 >> Running! Hang up with ^C! >> >> ^C >> >> Cannot set NIC flags! >> >> >> ls -alh /tmp/test >> total 9.0M >> drwxr-xr-x 2 daemon daemon 4.0K Jan 22 06:28 . >> drwxrwxrwt 13 root root 12K Jan 22 06:25 .. >> -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:26 snort.log.1358854003.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:27 snort.log.1358854011.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:27 snort.log.1358854026.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:27 snort.log.1358854044.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:28 snort.log.1358854064.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:28 snort.log.1358854085.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:28 snort.log.1358854090.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 22 06:28 snort.log.1358854091.pcap >> -rw-r--r-- 1 daemon daemon 448K Jan 22 06:28 snort.log.1358854113.pcap >> >> >> Looking forward to 0.5.8! > > > Good it works! > > Then if you want to use it, I suggest for now, you take a new Git snapshot. > > We're constantly on it, to get the open bullets from our TODO file zeroed > out. ;-) > > A new version will then be announced here as usual. > > >> On Mon, Jan 21, 2013 at 4:12 PM, Daniel Borkmann <[email protected]> >> wrote: >>> >>> On Mon, Jan 21, 2013 at 5:50 PM, Doug Burks <[email protected]> wrote: >>>> >>>> Excellent, thanks! >>> >>> >>> Should be fixed now. >>> >>>> On Mon, Jan 21, 2013 at 11:48 AM, Daniel Borkmann <[email protected]> >>>> wrote: >>>>> >>>>> On 01/21/2013 05:46 PM, Doug Burks wrote: >>>>>> >>>>>> >>>>>> After removing all set_ioprio_*() calls in the files pcap_mmap.c, >>>>>> pcap_rw.c, pcap_sg.c, it's now rotating properly: >>>>>> >>>>>> ./netsniff-ng -i eth0 -o /tmp/test/ -s --prefix snort.log. --interval >>>>>> 1MiB --user 1 --group 1 >>>>>> netsniff-ng 0.5.8-rc0 >>>>>> Running! Hang up with ^C! >>>>>> >>>>>> ^C >>>>>> >>>>>> Cannot set NIC flags! >>>>>> >>>>>> >>>>>> ls -alh /tmp/test >>>>>> total 5.0M >>>>>> drwxr-xr-x 2 daemon daemon 4.0K Jan 21 11:44 . >>>>>> drwxrwxrwt 13 root root 12K Jan 21 11:41 .. >>>>>> -rw-r--r-- 1 root root 1.1M Jan 21 11:44 >>>>>> snort.log.1358786624.pcap >>>>>> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 >>>>>> snort.log.1358786641.pcap >>>>>> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 >>>>>> snort.log.1358786656.pcap >>>>>> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 >>>>>> snort.log.1358786674.pcap >>>>>> -rw-r--r-- 1 daemon daemon 683K Jan 21 11:45 >>>>>> snort.log.1358786691.pcap >>>>> >>>>> >>>>> >>>>> Okay, so I will fix things up tonight and let you know when it's >>>>> upstream. >>>>> >>>>> >>>>>> On Mon, Jan 21, 2013 at 11:36 AM, Daniel Borkmann >>>>>> <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> On 01/21/2013 05:28 PM, Daniel Borkmann wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 01/21/2013 05:16 PM, Doug Burks wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Daniel, >>>>>>>>> >>>>>>>>> I tested using options similar to what we normally use. It writes >>>>>>>>> the >>>>>>>>> first pcap correctly (although I notice that the ownership is >>>>>>>>> root:root instead of daemon:daemon), but once the 1MiB interval has >>>>>>>>> been reached and netsniff-ng tries to rotate to the second pcap, it >>>>>>>>> fails when setting io prio. >>>>>>>>> >>>>>>>>> ./netsniff-ng -i eth0 -o /tmp/test/ -s --prefix snort.log. >>>>>>>>> --interval >>>>>>>>> 1MiB --user 1 --group 1 >>>>>>>>> netsniff-ng 0.5.8-rc0 >>>>>>>>> Running! Hang up with ^C! >>>>>>>>> >>>>>>>>> Failed to set io prio for pid! >>>>>>>>> >>>>>>>>> >>>>>>>>> ls -alh /tmp/test/ >>>>>>>>> total 3.3M >>>>>>>>> drwxr-xr-x 2 daemon daemon 4.0K Jan 21 11:08 . >>>>>>>>> drwxrwxrwt 13 root root 12K Jan 21 11:06 .. >>>>>>>>> -rw-r--r-- 1 root root 1.1M Jan 21 11:05 >>>>>>>>> snort.log.1358784314.pcap >>>>>>>>> -rw-r--r-- 1 daemon daemon 24 Jan 21 11:05 >>>>>>>>> snort.log.1358784337.pcap >>>>>>>>> >>>>>>>>> >>>>>>>>> Any ideas? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks for your feedback / tests! >>>>>>>> >>>>>>>> Ideas why it fails ... yes. >>>>>>>> >>>>>>>> The privileges are currently dropped right before entering the main >>>>>>>> loop, >>>>>>>> so >>>>>>>> the first file is still opened as root. Basically, the issue is that >>>>>>>> in >>>>>>>> the >>>>>>>> background netsniff-ng does some more automatic tuning that requires >>>>>>>> privileges >>>>>>>> (like setting the disc I/O scheduling policy), instead of just >>>>>>>> opening a >>>>>>>> file >>>>>>>> and that's it. >>>>>>>> >>>>>>>> So in the end, it's only useful for a couple of scenarios (for >>>>>>>> trafgen >>>>>>>> in >>>>>>>> general >>>>>>>> it's useful, netsniff-ng: for replaying a pcap, for recording a >>>>>>>> single >>>>>>>> pcap, ...), >>>>>>>> but probably not for the use case you have. Thus, I'm afraid, >>>>>>>> sticking >>>>>>>> with the >>>>>>>> setcap workaround might be the best solution. >>>>>>>> >>>>>>>> I don't see a value in shortly regaining privileges and then >>>>>>>> dropping >>>>>>>> them >>>>>>>> again >>>>>>>> (if that's actually possible with effective/read IDs); probably, it >>>>>>>> might >>>>>>>> be as >>>>>>>> good as not dropping them at all. >>>>>>>> >>>>>>>> Opinions? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> If it concerns only the ioprio, we could add another pcap function >>>>>>> that >>>>>>> gets >>>>>>> called >>>>>>> once before dropping the privs. >>>>>>> >>>>>>> Would it work for you, if you remove all set_ioprio_*() calls in the >>>>>>> files >>>>>>> pcap_mmap.c, >>>>>>> pcap_rw.c, pcap_sg.c ? >>>>>>> >>>>>>> If so, then we can do sth about it! ;-) >>>>>>> >>>>>>> Let me know. Thanks! >>>>>>> >>>>>>> >>>>>>>>> On Mon, Jan 21, 2013 at 5:01 AM, Daniel Borkmann >>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 01/19/2013 02:54 PM, Doug Burks wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks so much! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I'd be grateful if you can test it on your system. >>>>>>>>>> >>>>>>>>>> It should work for most cases. In netsniff-ng, I've seen that it >>>>>>>>>> exits >>>>>>>>>> non-gracefully >>>>>>>>>> when netsniff-ng is in promisc mode, since after destroying the >>>>>>>>>> mmap'ed >>>>>>>>>> rings it tries >>>>>>>>>> to put the card into non-promisc again, but has no rights for it >>>>>>>>>> (since >>>>>>>>>> we've dropped >>>>>>>>>> it). I don't think it makes much sense to regain privileged rights >>>>>>>>>> only >>>>>>>>>> for >>>>>>>>>> the >>>>>>>>>> cleanup work. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Fri, Jan 18, 2013 at 3:37 PM, Daniel Borkmann >>>>>>>>>>> <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Dec 2, 2012 at 10:11 PM, Doug Burks >>>>>>>>>>>> <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Well, since you asked... :) >>>>>>>>>>>>> >>>>>>>>>>>>> I know I can do the following to allow netsniff-ng to be run as >>>>>>>>>>>>> a >>>>>>>>>>>>> non-root user: >>>>>>>>>>>>> sudo setcap >>>>>>>>>>>>> cap_net_raw,cap_ipc_lock,cap_sys_admin,cap_net_admin=eip >>>>>>>>>>>>> netsniff-ng >>>>>>>>>>>>> >>>>>>>>>>>>> But it would be quite nice if netsniff-ng had internal support >>>>>>>>>>>>> for >>>>>>>>>>>>> dropping to a non-root user after opening the eth device. Many >>>>>>>>>>>>> other >>>>>>>>>>>>> pcap tools (such as daemonlogger, snort, suricata, etc.) >>>>>>>>>>>>> support >>>>>>>>>>>>> this >>>>>>>>>>>>> via command-line arguments like: >>>>>>>>>>>>> -u user -g group >>>>>>>>>>>>> OR >>>>>>>>>>>>> --user user --group group >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Done, after setup, you can now drop privileges in netsniff-ng >>>>>>>>>>>> and >>>>>>>>>>>> trafgen. >>>>>>>>>>>> >>>>>>>>>>>>> Thoughts? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your consideration! >>>>>>>>>>>>> >>>>>>>>>>>>> Doug >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Dec 2, 2012 at 3:50 PM, Daniel Borkmann >>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> By the way, if you have any other feature requests / wishes >>>>>>>>>>>>>> (besides >>>>>>>>>>>>>> the list in TODO) that might be useful for many users, let us >>>>>>>>>>>>>> know, >>>>>>>>>>>>>> and we'd be happy to further improve the toolkit. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sun, Dec 2, 2012 at 5:49 PM, Daniel Borkmann >>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, Dec 2, 2012 at 5:47 PM, Doug Burks >>>>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for the always fast response! Deploying the "tcpdump >>>>>>>>>>>>>>>> -dd" >>>>>>>>>>>>>>>> solution now. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If we have something new regarding bpfc, I'll announce it on >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> list >>>>>>>>>>>>>>> anyway. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sun, Dec 2, 2012 at 9:00 AM, Daniel Borkmann >>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Sun, Dec 2, 2012 at 2:55 PM, Doug Burks >>>>>>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Tue, Oct 30, 2012 at 10:41 AM, Daniel Borkmann >>>>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>>>> <snip> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -f /etc/nsm/$HOSTNAME-$INTERFACE/bpf-pcap.conf >>>>>>>>>>>>>>>>>>>> This should be the same option in netsniff-ng, but my >>>>>>>>>>>>>>>>>>>> understanding is >>>>>>>>>>>>>>>>>>>> that I'll need to convert my "human-readable" >>>>>>>>>>>>>>>>>>>> bpf-pcap.conf >>>>>>>>>>>>>>>>>>>> using >>>>>>>>>>>>>>>>>>>> "tcpdump -dd"? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yes, it you want to use filters and bpf-pcap.conf >>>>>>>>>>>>>>>>>>> contains >>>>>>>>>>>>>>>>>>> tcpdump-like filters, run them through "tcpdump -dd <my >>>>>>>>>>>>>>>>>>> filter>" > >>>>>>>>>>>>>>>>>>> out.ops and then pass out.ops to netsniff-ng via >>>>>>>>>>>>>>>>>>> "--filter >>>>>>>>>>>>>>>>>>> out.ops". >>>>>>>>>>>>>>>>>>> That's it; netsniff-ng will then automatically enable the >>>>>>>>>>>>>>>>>>> BPF >>>>>>>>>>>>>>>>>>> JIT >>>>>>>>>>>>>>>>>>> if >>>>>>>>>>>>>>>>>>> it's available in your kernel. This feature translates >>>>>>>>>>>>>>>>>>> BPF >>>>>>>>>>>>>>>>>>> filters >>>>>>>>>>>>>>>>>>> into architecture optimized machine opcodes within the >>>>>>>>>>>>>>>>>>> kernel. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> We've officially replaced daemonlogger with netsniff-ng >>>>>>>>>>>>>>>>>> and it >>>>>>>>>>>>>>>>>> appears >>>>>>>>>>>>>>>>>> to be working well! However, we haven't included BPF >>>>>>>>>>>>>>>>>> functionality >>>>>>>>>>>>>>>>>> yet, so I need to add that now. I can do what's described >>>>>>>>>>>>>>>>>> above, >>>>>>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>>> the FAQ also says: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Cool, I'm very happy about that! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> "If you try to create custom socket filters with tcpdump >>>>>>>>>>>>>>>>>> -dd, >>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>> to edit the ret opcode (0x6) of the resulting filter, >>>>>>>>>>>>>>>>>> otherwise >>>>>>>>>>>>>>>>>> your >>>>>>>>>>>>>>>>>> payload will be cut off: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 0x6, 0, 0, 0xFFFFFFFF instead of 0x6, 0, 0, 0x00000060 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The Linux kernel now takes skb->len instead of 0xFFFFFFFF. >>>>>>>>>>>>>>>>>> If >>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>> do >>>>>>>>>>>>>>>>>> not change it, the kernel will take 0x00000060 as buffer >>>>>>>>>>>>>>>>>> length >>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>> packets larger than 96 Byte will be cut off (filled with >>>>>>>>>>>>>>>>>> zero >>>>>>>>>>>>>>>>>> Bytes)! >>>>>>>>>>>>>>>>>> It's a bug in libpcaps filter compiler. Detailed >>>>>>>>>>>>>>>>>> information >>>>>>>>>>>>>>>>>> about >>>>>>>>>>>>>>>>>> this issue can be found on our blog post." >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The linked blog post is no longer available. So is this >>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>> issue I >>>>>>>>>>>>>>>>>> need to be concerned about? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Actually not anymore. I use Fedora and the tcpdump version >>>>>>>>>>>>>>>>> there >>>>>>>>>>>>>>>>> outputs: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> # tcpdump -dd ip >>>>>>>>>>>>>>>>> tcpdump: WARNING: eth0: no IPv4 address assigned >>>>>>>>>>>>>>>>> { 0x28, 0, 0, 0x0000000c }, >>>>>>>>>>>>>>>>> { 0x15, 0, 1, 0x00000800 }, >>>>>>>>>>>>>>>>> { 0x6, 0, 0, 0x0000ffff }, >>>>>>>>>>>>>>>>> { 0x6, 0, 0, 0x00000000 }, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> So they have changed this from 0x00000060 into 0x0000ffff. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For bpfc itself, I didn't have time to finish the >>>>>>>>>>>>>>>>> high-level >>>>>>>>>>>>>>>>> compiler, >>>>>>>>>>>>>>>>> yet. We have an assembler-like compiler where you can also >>>>>>>>>>>>>>>>> create >>>>>>>>>>>>>>>>> filters with, but for usability you can use the method >>>>>>>>>>>>>>>>> described >>>>>>>>>>>>>>>>> above, of course. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Doug Burks >>>>>>>>>>>>>>>> http://securityonion.blogspot.com >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Doug Burks >>>>>>>>>>>>> http://securityonion.blogspot.com >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Doug Burks >>>> http://securityonion.blogspot.com >>>> >>>> -- >>>> >>>> >>> >>> -- >>> >>> >> >> >> > > -- > >
--
