Excellent, thanks! Doug On Mon, Jan 21, 2013 at 11:48 AM, Daniel Borkmann <[email protected]> wrote: > On 01/21/2013 05:46 PM, Doug Burks wrote: >> >> After removing all set_ioprio_*() calls in the files pcap_mmap.c, >> pcap_rw.c, pcap_sg.c, it's now rotating properly: >> >> ./netsniff-ng -i eth0 -o /tmp/test/ -s --prefix snort.log. --interval >> 1MiB --user 1 --group 1 >> netsniff-ng 0.5.8-rc0 >> Running! Hang up with ^C! >> >> ^C >> >> Cannot set NIC flags! >> >> >> ls -alh /tmp/test >> total 5.0M >> drwxr-xr-x 2 daemon daemon 4.0K Jan 21 11:44 . >> drwxrwxrwt 13 root root 12K Jan 21 11:41 .. >> -rw-r--r-- 1 root root 1.1M Jan 21 11:44 snort.log.1358786624.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 snort.log.1358786641.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 snort.log.1358786656.pcap >> -rw-r--r-- 1 daemon daemon 1.1M Jan 21 11:44 snort.log.1358786674.pcap >> -rw-r--r-- 1 daemon daemon 683K Jan 21 11:45 snort.log.1358786691.pcap > > > Okay, so I will fix things up tonight and let you know when it's upstream. > > >> On Mon, Jan 21, 2013 at 11:36 AM, Daniel Borkmann <[email protected]> >> wrote: >>> >>> On 01/21/2013 05:28 PM, Daniel Borkmann wrote: >>>> >>>> >>>> On 01/21/2013 05:16 PM, Doug Burks wrote: >>>>> >>>>> >>>>> Hi Daniel, >>>>> >>>>> I tested using options similar to what we normally use. It writes the >>>>> first pcap correctly (although I notice that the ownership is >>>>> root:root instead of daemon:daemon), but once the 1MiB interval has >>>>> been reached and netsniff-ng tries to rotate to the second pcap, it >>>>> fails when setting io prio. >>>>> >>>>> ./netsniff-ng -i eth0 -o /tmp/test/ -s --prefix snort.log. --interval >>>>> 1MiB --user 1 --group 1 >>>>> netsniff-ng 0.5.8-rc0 >>>>> Running! Hang up with ^C! >>>>> >>>>> Failed to set io prio for pid! >>>>> >>>>> >>>>> ls -alh /tmp/test/ >>>>> total 3.3M >>>>> drwxr-xr-x 2 daemon daemon 4.0K Jan 21 11:08 . >>>>> drwxrwxrwt 13 root root 12K Jan 21 11:06 .. >>>>> -rw-r--r-- 1 root root 1.1M Jan 21 11:05 snort.log.1358784314.pcap >>>>> -rw-r--r-- 1 daemon daemon 24 Jan 21 11:05 snort.log.1358784337.pcap >>>>> >>>>> >>>>> Any ideas? >>>> >>>> >>>> >>>> Thanks for your feedback / tests! >>>> >>>> Ideas why it fails ... yes. >>>> >>>> The privileges are currently dropped right before entering the main >>>> loop, >>>> so >>>> the first file is still opened as root. Basically, the issue is that in >>>> the >>>> background netsniff-ng does some more automatic tuning that requires >>>> privileges >>>> (like setting the disc I/O scheduling policy), instead of just opening a >>>> file >>>> and that's it. >>>> >>>> So in the end, it's only useful for a couple of scenarios (for trafgen >>>> in >>>> general >>>> it's useful, netsniff-ng: for replaying a pcap, for recording a single >>>> pcap, ...), >>>> but probably not for the use case you have. Thus, I'm afraid, sticking >>>> with the >>>> setcap workaround might be the best solution. >>>> >>>> I don't see a value in shortly regaining privileges and then dropping >>>> them >>>> again >>>> (if that's actually possible with effective/read IDs); probably, it >>>> might >>>> be as >>>> good as not dropping them at all. >>>> >>>> Opinions? >>> >>> >>> >>> If it concerns only the ioprio, we could add another pcap function that >>> gets >>> called >>> once before dropping the privs. >>> >>> Would it work for you, if you remove all set_ioprio_*() calls in the >>> files >>> pcap_mmap.c, >>> pcap_rw.c, pcap_sg.c ? >>> >>> If so, then we can do sth about it! ;-) >>> >>> Let me know. Thanks! >>> >>> >>>>> On Mon, Jan 21, 2013 at 5:01 AM, Daniel Borkmann <[email protected]> >>>>> wrote: >>>>>> >>>>>> >>>>>> On 01/19/2013 02:54 PM, Doug Burks wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks so much! >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I'd be grateful if you can test it on your system. >>>>>> >>>>>> It should work for most cases. In netsniff-ng, I've seen that it exits >>>>>> non-gracefully >>>>>> when netsniff-ng is in promisc mode, since after destroying the >>>>>> mmap'ed >>>>>> rings it tries >>>>>> to put the card into non-promisc again, but has no rights for it >>>>>> (since >>>>>> we've dropped >>>>>> it). I don't think it makes much sense to regain privileged rights >>>>>> only >>>>>> for >>>>>> the >>>>>> cleanup work. >>>>>> >>>>>> >>>>>>> On Fri, Jan 18, 2013 at 3:37 PM, Daniel Borkmann >>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Dec 2, 2012 at 10:11 PM, Doug Burks <[email protected]> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Well, since you asked... :) >>>>>>>>> >>>>>>>>> I know I can do the following to allow netsniff-ng to be run as a >>>>>>>>> non-root user: >>>>>>>>> sudo setcap >>>>>>>>> cap_net_raw,cap_ipc_lock,cap_sys_admin,cap_net_admin=eip >>>>>>>>> netsniff-ng >>>>>>>>> >>>>>>>>> But it would be quite nice if netsniff-ng had internal support for >>>>>>>>> dropping to a non-root user after opening the eth device. Many >>>>>>>>> other >>>>>>>>> pcap tools (such as daemonlogger, snort, suricata, etc.) support >>>>>>>>> this >>>>>>>>> via command-line arguments like: >>>>>>>>> -u user -g group >>>>>>>>> OR >>>>>>>>> --user user --group group >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Done, after setup, you can now drop privileges in netsniff-ng and >>>>>>>> trafgen. >>>>>>>> >>>>>>>>> Thoughts? >>>>>>>>> >>>>>>>>> Thanks for your consideration! >>>>>>>>> >>>>>>>>> Doug >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Dec 2, 2012 at 3:50 PM, Daniel Borkmann >>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> By the way, if you have any other feature requests / wishes >>>>>>>>>> (besides >>>>>>>>>> the list in TODO) that might be useful for many users, let us >>>>>>>>>> know, >>>>>>>>>> and we'd be happy to further improve the toolkit. >>>>>>>>>> >>>>>>>>>> On Sun, Dec 2, 2012 at 5:49 PM, Daniel Borkmann >>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Dec 2, 2012 at 5:47 PM, Doug Burks <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks for the always fast response! Deploying the "tcpdump >>>>>>>>>>>> -dd" >>>>>>>>>>>> solution now. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>>> If we have something new regarding bpfc, I'll announce it on the >>>>>>>>>>> list >>>>>>>>>>> anyway. >>>>>>>>>>> >>>>>>>>>>>> On Sun, Dec 2, 2012 at 9:00 AM, Daniel Borkmann >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Dec 2, 2012 at 2:55 PM, Doug Burks >>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Oct 30, 2012 at 10:41 AM, Daniel Borkmann >>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>> <snip> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -f /etc/nsm/$HOSTNAME-$INTERFACE/bpf-pcap.conf >>>>>>>>>>>>>>>> This should be the same option in netsniff-ng, but my >>>>>>>>>>>>>>>> understanding is >>>>>>>>>>>>>>>> that I'll need to convert my "human-readable" bpf-pcap.conf >>>>>>>>>>>>>>>> using >>>>>>>>>>>>>>>> "tcpdump -dd"? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes, it you want to use filters and bpf-pcap.conf contains >>>>>>>>>>>>>>> tcpdump-like filters, run them through "tcpdump -dd <my >>>>>>>>>>>>>>> filter>" > >>>>>>>>>>>>>>> out.ops and then pass out.ops to netsniff-ng via "--filter >>>>>>>>>>>>>>> out.ops". >>>>>>>>>>>>>>> That's it; netsniff-ng will then automatically enable the BPF >>>>>>>>>>>>>>> JIT >>>>>>>>>>>>>>> if >>>>>>>>>>>>>>> it's available in your kernel. This feature translates BPF >>>>>>>>>>>>>>> filters >>>>>>>>>>>>>>> into architecture optimized machine opcodes within the >>>>>>>>>>>>>>> kernel. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> We've officially replaced daemonlogger with netsniff-ng and it >>>>>>>>>>>>>> appears >>>>>>>>>>>>>> to be working well! However, we haven't included BPF >>>>>>>>>>>>>> functionality >>>>>>>>>>>>>> yet, so I need to add that now. I can do what's described >>>>>>>>>>>>>> above, >>>>>>>>>>>>>> but >>>>>>>>>>>>>> the FAQ also says: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Cool, I'm very happy about that! >>>>>>>>>>>>> >>>>>>>>>>>>>> "If you try to create custom socket filters with tcpdump -dd, >>>>>>>>>>>>>> you >>>>>>>>>>>>>> have >>>>>>>>>>>>>> to edit the ret opcode (0x6) of the resulting filter, >>>>>>>>>>>>>> otherwise >>>>>>>>>>>>>> your >>>>>>>>>>>>>> payload will be cut off: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 0x6, 0, 0, 0xFFFFFFFF instead of 0x6, 0, 0, 0x00000060 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The Linux kernel now takes skb->len instead of 0xFFFFFFFF. If >>>>>>>>>>>>>> you >>>>>>>>>>>>>> do >>>>>>>>>>>>>> not change it, the kernel will take 0x00000060 as buffer >>>>>>>>>>>>>> length >>>>>>>>>>>>>> and >>>>>>>>>>>>>> packets larger than 96 Byte will be cut off (filled with zero >>>>>>>>>>>>>> Bytes)! >>>>>>>>>>>>>> It's a bug in libpcaps filter compiler. Detailed information >>>>>>>>>>>>>> about >>>>>>>>>>>>>> this issue can be found on our blog post." >>>>>>>>>>>>>> >>>>>>>>>>>>>> The linked blog post is no longer available. So is this an >>>>>>>>>>>>>> issue I >>>>>>>>>>>>>> need to be concerned about? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Actually not anymore. I use Fedora and the tcpdump version >>>>>>>>>>>>> there >>>>>>>>>>>>> outputs: >>>>>>>>>>>>> >>>>>>>>>>>>> # tcpdump -dd ip >>>>>>>>>>>>> tcpdump: WARNING: eth0: no IPv4 address assigned >>>>>>>>>>>>> { 0x28, 0, 0, 0x0000000c }, >>>>>>>>>>>>> { 0x15, 0, 1, 0x00000800 }, >>>>>>>>>>>>> { 0x6, 0, 0, 0x0000ffff }, >>>>>>>>>>>>> { 0x6, 0, 0, 0x00000000 }, >>>>>>>>>>>>> >>>>>>>>>>>>> So they have changed this from 0x00000060 into 0x0000ffff. >>>>>>>>>>>>> >>>>>>>>>>>>> For bpfc itself, I didn't have time to finish the high-level >>>>>>>>>>>>> compiler, >>>>>>>>>>>>> yet. We have an assembler-like compiler where you can also >>>>>>>>>>>>> create >>>>>>>>>>>>> filters with, but for usability you can use the method >>>>>>>>>>>>> described >>>>>>>>>>>>> above, of course. >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Doug Burks >>>>>>>>>>>> http://securityonion.blogspot.com >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Doug Burks >>>>>>>>> http://securityonion.blogspot.com >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>> >>> -- >>> >>> >> >> >> > > -- > >
-- Doug Burks http://securityonion.blogspot.com --
