Re: Packet Filter: how to keep device names on hardware failure?
On Mon, Nov 10, 2008 at 4:13 AM, Harald Dunkel [EMAIL PROTECTED] wrote: Hi Theo, Theo de Raadt wrote: This appears to be a fairly simple change. Does it sound reasonable to people with more knowledge of OpenBSD networking? No, it is not reasonble. You are inventing problems at a very high level just because some very low level pci-related bug is making some of your devices not reliably show themselves. Surely you are very close to the details of OpenBSD kernel development, but maybe it is more helpful to keep a little bit of distance. I see a system that introduces names for some of its parts on its own, using some internal rules that I cannot influence. These names are not well-defined. Surely they are not supposed to change over time, but on certain events they can. Some of these events I cannot influence, either. The problem is, that I am forced to reference these names in the configuration files for a highly important part of the system, where a change of these names could be fatal. If the names have changed, it is because your hardware has changed or failed. That type of problem should demand your attention anyways. With some thought and prior knowledge, you may be able build and configure your system so that partial device failure will not result in the situation you describe. It is not always possible though. Surely I am not asking to drop the traditional naming scheme immediately, and replace it by something else. All I am asking for is to think about it, how this could be improved. Did you not read the earlier replies? People may be thinking about it, but it is not the fatal flaw you build it up to be. As others have suggested, there are already networking tools in OpenBSD that can help you provide link redundancy by using a second computer. Your improvement may happen someday but from what I've read in this thread, I wouldn't hold your breath waiting for it. --david
Re: Packet Filter: how to keep device names on hardware failure?
Jussi Peltola wrote: I see no problem in setting interface groups based on mac address. You should be able to hack a suitable script to do that in a few minutes. AFAICS brconfig does not support group names. Regards Harri
Re: Packet Filter: how to keep device names on hardware failure?
Hi Theo, Theo de Raadt wrote: This appears to be a fairly simple change. Does it sound reasonable to people with more knowledge of OpenBSD networking? No, it is not reasonble. You are inventing problems at a very high level just because some very low level pci-related bug is making some of your devices not reliably show themselves. Surely you are very close to the details of OpenBSD kernel development, but maybe it is more helpful to keep a little bit of distance. I see a system that introduces names for some of its parts on its own, using some internal rules that I cannot influence. These names are not well-defined. Surely they are not supposed to change over time, but on certain events they can. Some of these events I cannot influence, either. The problem is, that I am forced to reference these names in the configuration files for a highly important part of the system, where a change of these names could be fatal. Surely I am not asking to drop the traditional naming scheme immediately, and replace it by something else. All I am asking for is to think about it, how this could be improved. Many thanx Harri
Re: Packet Filter: how to keep device names on hardware failure?
What about VLANs? At least I wouldn't torture myself with ancient/cheap switches that don't support them. Of course, then you have to worry about the switch breaking or getting its config reset...
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, 7 Nov 2008, johan beisser wrote: On Nov 7, 2008, at 9:44 AM, Dave Anderson wrote: Perhaps most of these issues could be dealt with by changing the network configuration procedure to have a hierarchy of interface-configuration files rather than just hostname.interface-name. If hostname.mac were used if the hardware MAC matches, then hostname.interface-name, then (say) hostname.only if there's only one NIC found, the sysadmin could assign interfaces to groups and use those group names everywhere, and so not need to use the actual interface names at all. This appears to be a fairly simple change. Does it sound reasonable to people with more knowledge of OpenBSD networking? It's not a simple change. Having now looked at /etc/netstart, it's clearly not as simple as I'd hoped -- but it doesn't look all that difficult. The only issue I don't (yet) see a solution for is how to get the original hardware MAC address for an interface (rather than the current MAC address, which appears to be what ifconfig reports). I could parse the dmesg from the most recent boot, but that feels wrong -- especially since I'm not certain that that information will always be available, complete and unaltered. Dave -- Dave Anderson [EMAIL PROTECTED]
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, Nov 7, 2008 at 1:30 PM, Harald Dunkel [EMAIL PROTECTED] wrote: In the bad configuration the NIC with 00:30:48:d2:9a:06 is called em2, in the good one it is called em4. Maybe you can imagine how PF screws up, if this NIC would have been physically connected to the Internet. Surely it is unusual that a NIC disappears somehow. Maybe there is something wrong with my hardware, but this can always happen. I would like to have a secure setup even if there is a hardware failure. what keeps you from writing a script that would be called from the end of /etc/netstart; the script would check whether the initialized network interfaces match those described by a predefined table? in case of failure it would react somehow... you could also put in a NIC of some other type that would always be named the same (e.g. xl0) that would be an interface used for reporting the failure with those emX?
Re: Packet Filter: how to keep device names on hardware failure?
Denis Doroshenko [EMAIL PROTECTED] writes: what keeps you from writing a script that would be called from the end of /etc/netstart; the script would check whether the initialized network interfaces match those described by a predefined table? in case of failure it would react somehow... Then again, given the 'failure is not an option' scenario, any sane network design would mean you most likely have a multiply redundant CARP'd setup in place, so a hardware failure like the one described on one box would simply mean the machine would take itself out of the running, one of the backups would take over and your friendly robot helper would be paging you to replace the failed hardware at your earliest opportunity. By all means nothing stops you from writing script magic, but the tools already in your OpenBSD base system lets you solve these situations quite admirably and in several differen ways already. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: Packet Filter: how to keep device names on hardware failure?
Peter N. M. Hansteen wrote: Denis Doroshenko [EMAIL PROTECTED] writes: what keeps you from writing a script that would be called from the end of /etc/netstart; the script would check whether the initialized network interfaces match those described by a predefined table? in case of failure it would react somehow... Then again, given the 'failure is not an option' scenario, any sane network design would mean you most likely have a multiply redundant CARP'd setup in place, so a hardware failure like the one described on one box would simply mean the machine would take itself out of the running, one of the backups would take over and your friendly robot helper would be paging you to replace the failed hardware at your earliest opportunity. By all means nothing stops you from writing script magic, but the tools already in your OpenBSD base system lets you solve these situations quite admirably and in several differen ways already. If you actually require fault tolerance, this is the best advice so far. Your devices are ordered as you expect them to be, your rule base is in a known good state. The system uses supported features making upgrades simple, as well as leaving off the sort of site specific quirks that can make inheriting a site so challenging.
Re: Packet Filter: how to keep device names on hardware failure?
Rod Whitworth wrote: ... Let's look at this a little more analytically: My firewall is a Soekris 4801 with sis0, sis1 and sis2. sis0 is the 0utside (ADSL) sis1 is the 1nside (LAN) sis2 is the 2erver LAN heh. I gotta remember that naming/numbering convention, I like it! If 0 fails the other two move up the table. Risk = zero. If 1 fails the users holler No service! and the servers won't be compromised because they will now be connected to sis2 promoted to be sis1 and their default route won't be available and incoming traffic can't get to them either. Now, what was the problem again? With all the interfaces below the failure moving up the table there will be address mismatches = no traffic. I see no reason to panic. Maybe I'm too tired after being up really late replacing a faulty modem and I forgot to turn off NAT in the new one so my sleepy eyes missed the fact that I needed to test more than browsing from the LAN to make sure my servers were reachable. 8-(( 8 snip rest of story. Yeah, maybe I'm missing something too, but I'm not really thinking of a situation where this would really be a risk of anything other than downtime. And if chunks of your firewall aren't working, that's downtime. Usually, if you plug the wrong things into the wrong port, it just doesn't work. Different ports are usually on different subnets. If you really have a situation where this is a real risk and not just a silly panic over nothing, a solution is simple: * your /etc/pf.conf file just contains a block in all, and a pass out all from just the firewall to the outside networks. * in rc.local, you stick a script which tests things however you want them to be. Maybe you count the NICs, maybe you compare their MAC addresses to what you expect them to be, etc. Whatever makes you happy or is appropriate for your configuration. * IF you are happy, you do a pfctl -f /etc/prodpf.conf or similar, and put your production rules in there. Maybe even only activate forwarding if the test passes. IF the system is missing pieces, maybe you load up an ssh in only ruleset so someone can get to the box to look at what went wrong, but the firewall stays otherwise inert. Document the heck out of it, including in pf.conf saying, real production rules are over THERE... Note that this requires modifying no system files, so your upgrade process remains simple. I think that would be a lot saner for what seems to be a very special case than any of the let's follow Linux or Solaris's lead crap. I've used those, I'm completely unimpressed. The primary reason they suck is complexity; the people who claim they understand Linux and Solaris don't seem to be able to explain why they do what they do or fix it when they do it wrong, forget mere mortals. They just work around oddities. OpenBSD's rules for NIC naming are quite simple. There are cases where they will annoy the heck out of you, but it is easy to see WHY they are doing funny things, and easy to fix when they do. When my firewall blows out when I'm on vacation, I want to be able to tell someone over the phone, unplug the production machine, keeping careful track of what cable comes out of which port, plug them into the same port on the spare machine. Pull the disk out of the old machine, plug it into the spare machine. Turn it on, see you when I get back. Start strapping ports to physical addresses, you create a management nightmare, and something that probably only you will ever be able to maintain. Not good. Nick.
Re: Packet Filter: how to keep device names on hardware failure?
Hi folks, Harald Dunkel wrote: Question: How can I make sure that em2 doesn't become em0 if my dual-port NIC dies? This would be fatal for my firewall setup. At least the antispoof rules _must_ be bound to the network devices. Sorry to wake this thread up again, but this problem is a severe security risk. IMHO it is unacceptable that a hardware failure on one NIC of a firewall can put the whole network at risk, just because the mapping between NICs and interface names gets mixed up, and PF suddenly treats the Internet as a subnet of the company LAN. The workarounds posted here just show that OpenBSD's native functionality is insufficient. If you are planning a version 5.0, then it would be very nice if you could address this problem. Many thanx Harri
Re: Packet Filter: how to keep device names on hardware failure?
Did you fill a bug report? -- Aram Havarneanu
Re: Packet Filter: how to keep device names on hardware failure?
Peter N. M. Hansteen schrieb:
Harald Dunkel [EMAIL PROTECTED] writes:
maybe you can use something like this in your script:
int_if=xx:xx:xx:xx:xx:xx
ext_if=yy:yy:yy:yy:yy:yy
int_if=`ifconfig|grep -e $int_if|awk '{print $1}'`
ext_if=`ifconfig|grep -e $ext_if|awk '{print $1}'`
This will not directly work on OpenBSD as the output of ifconfig is not
the same as on Linux and I used it on Linux as some linux kernels
doesn't enumerated the network interfaces in the same order at each boot
and they were all called ethx, so rebooting without scanning for
mac-adresses was a real mess.
Surely we assume that nobody fakes the mac.
guido
Re: Packet Filter: how to keep device names on hardware failure?
On 13:43:11 Nov 07, Guido Tschakert wrote: Surely we assume that nobody fakes the mac. I could be wrong but I don't think it possible to fake the MAC reported in dmesg(8). ifconfig can fake MAC address but this should be unique since it is reported by the NIC whilst probing. -Girish
Re: Packet Filter: how to keep device names on hardware failure?
Harald Dunkel [EMAIL PROTECTED] writes: I can post 2 dmesg logs of the same machine with the NIC names mixed up. Somehow 2 NICs disappeared on a reboot. On the next reboot they were back. Attached is the diff. Dodgy hardware does lead to problems, certainly. The basic problem here is that the system enumerates only the hardware it is able to contact and identify, and units of the same type are assigned an identifier equal to driver name plus a sequence number. Unless we make some other unique identifier part of the way PF evaluates rules (the MAC address comes to mind, but that too can be changed in any modern operating system), there is no quick fix, other than rewriting your rule set so it avoids 'on' criteria and other hardware specifics wherever possible. The other immediately available workaround would be to make sure you build your system so no two network interfaces will use the same driver. That earns you the interesting tradeoff of handling a number of suppliers roughly equal to the number of interfaces installed in case of hardware failure, with the likely consequence of having to stock N times as many different spares in case of failures. Surely it is unusual that a NIC disappears somehow. Maybe there is something wrong with my hardware, but this can always happen. I would like to have a secure setup even if there is a hardware failure. Hardware failures will occur occasionally. Depending on your local security assessment there are a number of variables at play here, and it can be convincingly argued that you should have spare parts on hand, along with a proper monitoring regime and procedures that will have you replace faulty hardware before the scenario you describe occurs. On the other hand, it would be interesting to see how competing systems handle this scenario, if at all. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: Packet Filter: how to keep device names on hardware failure?
Peter N. M. Hansteen wrote: Harald Dunkel [EMAIL PROTECTED] writes: Sorry to wake this thread up again, but this problem is a severe security risk. IMHO it is unacceptable that a hardware failure on one NIC of a firewall can put the whole network at risk, just because the mapping between NICs and interface names gets mixed up, and PF suddenly treats the Internet as a subnet of the company LAN. Semi-random reordering of network interfaces would be a severe problem, no doubt. However, my hazy memory was that reordering would not occur as you describe, but ICBW, please correct me if this has actually been demonstrated to happen. I can post 2 dmesg logs of the same machine with the NIC names mixed up. Somehow 2 NICs disappeared on a reboot. On the next reboot they were back. Attached is the diff. In the bad configuration the NIC with 00:30:48:d2:9a:06 is called em2, in the good one it is called em4. Maybe you can imagine how PF screws up, if this NIC would have been physically connected to the Internet. Surely it is unusual that a NIC disappears somehow. Maybe there is something wrong with my hardware, but this can always happen. I would like to have a secure setup even if there is a hardware failure. Regards Harri === --- dmesg4 2008-11-07 10:05:27.0 +0100 +++ dmesg6 2008-11-07 10:06:40.0 +0100 @@ -1,4 +1,4 @@ -# dmesg +# dmesg OpenBSD 4.3 (RAID.MP) #0: Wed Oct 1 14:07:35 CEST 2008 [EMAIL PROTECTED]:/usr/src/sys/arch/amd64/compile/RAID.MP real mem = 3487612928 (3326MB) @@ -14,7 +14,7 @@ acpihpet0 at acpi0: 14318179 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) -cpu0: Intel(R) Xeon(R) CPU E3110 @ 3.00GHz, 2992.85 MHz +cpu0: Intel(R) Xeon(R) CPU E3110 @ 3.00GHz, 2992.89 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG cpu0: 6MB 64b/line 16-way L2 cache cpu0: apic clock running at 332MHz @@ -64,12 +64,14 @@ em1 at pci6 dev 0 function 1 Intel PRO/1000 QP (82571EB) rev 0x06: apic 2 int 18 (irq 11), address 00:15:17:91:5a:80 ppb6 at pci5 dev 4 function 0 vendor IDT, unknown product 0x8018 rev 0x0e pci7 at ppb6 bus 8 +em2 at pci7 dev 0 function 0 Intel PRO/1000 QP (82571EB) rev 0x06: apic 2 int 17 (irq 10), address 00:15:17:91:5a:83 +em3 at pci7 dev 0 function 1 Intel PRO/1000 QP (82571EB) rev 0x06: apic 2 int 16 (irq 5), address 00:15:17:91:5a:82 ppb7 at pci0 dev 28 function 4 Intel 82801I PCIE rev 0x02: apic 2 int 16 (irq 5) pci8 at ppb7 bus 13 -em2 at pci8 dev 0 function 0 Intel PRO/1000MT (82573E) rev 0x03: apic 2 int 16 (irq 5), address 00:30:48:d2:9a:06 +em4 at pci8 dev 0 function 0 Intel PRO/1000MT (82573E) rev 0x03: apic 2 int 16 (irq 5), address 00:30:48:d2:9a:06 ppb8 at pci0 dev 28 function 5 Intel 82801I PCIE rev 0x02: apic 2 int 17 (irq 11) pci9 at ppb8 bus 15 -em3 at pci9 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: apic 2 int 17 (irq 10), address 00:30:48:d2:9a:07 +em5 at pci9 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: apic 2 int 17 (irq 10), address 00:30:48:d2:9a:07 uhci3 at pci0 dev 29 function 0 Intel 82801I USB rev 0x02: apic 2 int 23 (irq 10) uhci4 at pci0 dev 29 function 1 Intel 82801I USB rev 0x02: apic 2 int 22 (irq 11) uhci5 at pci0 dev 29 function 2 Intel 82801I USB rev 0x02: apic 2 int 18 (irq 11) @@ -91,7 +93,7 @@ ichiic0 at pci0 dev 31 function 3 Intel 82801I SMBus rev 0x02: apic 2 int 17 (irq 10) iic0 at ichiic0 lm1 at iic0 addr 0x2d: W83627HF -iic0: addr 0x2f 05=61 06=fb 07=61 08=fb 09=64 0a=64 0b=5e 0c=c8 0d=a3 0e=7b 0f=13 10=94 11=9d 12=89 13=ff 14=21 15=72 16=cf 17=7b 18=cb 19=cd 1a=c9 1b=01 1c=28 1d=9c 1e=80 1f=80 20=21 21=20 23=0f 25=0f 27=0f 29=0f 2b=0f 2d=0f 2f=02 30=ee 31=0f 33=0f 35=0f 3b=ff 3c=ff 3d=ff 3e=ff 3f=ff 40=09 46=f7 47=ef 48=ff 49=7e 4a=3f 4d=fc 4e=0e 50=06 51=02 52=01 58=28 59=01 5c=1f 5e=ff 5f=03 60=ac 61=73 62=ff 64=a5 65=7c 66=ff 67=ff 68=3f 6a=2a 6b=19 6c=7c 6d=65 6e=e3 6f=b9 70=8a 71=70 72=e5 73=bb 74=e5 75=bb 76=e3 77=b9 78=53 79=4e 7a=55 7b=50 7c=53 7d=4e 7e=55 7f=50 80=53 81=4e 82=55 83=50 84=53 85=4e 86=55 87=50 88=32 89=2d 8a=55 8b=50 8c=32 8d=2d 8e=55 8f=50 90=07 91=68 92=07 93=68 94=07 95=68 96=07 97=68 98=07 99=68 9a=07 9b=68 9c=07 9d=68 9e=07 9f=68 a0=07 a1=68 a2=07 a3=68 a4=ff a5=ff a6=ff a7=ff ae=ff af=ff b2=3f b3=3f b6=3f b7=3f b8=3f b9=3f ba=3f bb=89 bc=89 bd=89 be=89 bf=89 c0=89 c1=89 c2=89 c3=02 c4=03 c5=7f c6=ff c9=ff ca=ff cb=ff cc=ff cd=ff ce=ff cf=ff d0=10 d1=64 d2=64 d 3=64 d4=64 d6=e0 d7=ff d8=f1 da=80 db=01 dc=80 dd=01 de=80 df=01 e0=bb e1=c0 e2=82 e3=ff e4=80 e5=6e e6=fd e7=13 e8=11 e9=10 ea=20 eb=ea ec=ff ed=ff ee=ff ef=ff f6=60 f7=80 f8=1b fa=ff fd=10 words 00=00ff 01=00ff 02=00ff 03=00ff 04=00ff 05=61ff 06=fbff 07=61ff 08=ebff 09=64ff 0a=64ff 0b=5eff 0c=c8ff 0d=a3ff
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, Nov 07, 2008 at 01:22:08PM +0100, Peter N. M. Hansteen wrote: Unless we make some other unique identifier part of the way PF evaluates rules (the MAC address comes to mind, but that too can be changed in any modern operating system), there is no quick fix, other than rewriting your rule set so it avoids 'on' criteria and other hardware specifics wherever possible. Free advice without a patch is, of course, worth the price, but: If there was a way of recording the MAC address assigned to each interface by the kernel, then on a subsequent reboot could the kernel read that file to ensure that previously seen interfaces were assigend the same number? On Linux (Debian), interfaces are all ethx no matter what vendor. The udev system is supposed to record coresponding MAC in a persistant rules file to prevent this problem. Of course, this doesn't seem to work on some boxes for drives, so that, for example, a boot fails if a USB stick is plugged in because it may be assigned the /dev/sdx that is supposed to be the root drive. This prompts hacks of mounting with LABEL or UUID. Perhaps pf could be configured with MAC addres instead of interface id. Sure the MAC address could be changed by the sysadmin, but does it get changed at random by the OS? Just some early-morning thoughts, for what their worth. Doug.
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, Nov 7, 2008 at 11:33 AM, Douglas A. Tutty [EMAIL PROTECTED] wrote: On Fri, Nov 07, 2008 at 01:22:08PM +0100, Peter N. M. Hansteen wrote: Unless we make some other unique identifier part of the way PF evaluates rules (the MAC address comes to mind, but that too can be changed in any modern operating system), there is no quick fix, other than rewriting your rule set so it avoids 'on' criteria and other hardware specifics wherever possible. I don't see the ability to change a MAC address as a problem. Someone would need to get root access inside the router to make this change. So, since the bad guy is already root, there's not many things to be done to protect the machine... Free advice without a patch is, of course, worth the price, but: I'll take this words as mine as I discuss this matter in this message. If there was a way of recording the MAC address assigned to each interface by the kernel, then on a subsequent reboot could the kernel read that file to ensure that previously seen interfaces were assigend the same number? On Linux (Debian), interfaces are all ethx no matter what vendor. The udev system is supposed to record coresponding MAC in a persistant rules file to prevent this problem. Of course, this doesn't seem to work on some boxes for drives, so that, for example, a boot fails if a USB stick is plugged in because it may be assigned the /dev/sdx that is supposed to be the root drive. This prompts hacks of mounting with LABEL or UUID. In linux, there's a utility called ifrename. I had to use it in a massive linux installation once. The guys performing customer support were dumb enough to not learn the ethX addressing. I've used ifrename to change the names such as eth0, eth1 and eth2 to wan, lan and dmz. I really would like to have this kind of support on OpenBSD, but the NIC naming schemas of Linux and *BSD have huge differences. Perhaps pf could be configured with MAC addres instead of interface id. Sure the MAC address could be changed by the sysadmin, but does it get changed at random by the OS? One idea I had a couple years ago envolves changing the way the interface drivers are loaded in the kernel. Now, the schema is static. Probably translating it to a dynamic one could have some gains. My idea was to provide a mapping (or alias) to a network card based on its MAC address, just like ifrename on linux. One could use a file in /etc/ (say, /etc/ifrename.conf) to configure the system as follows: = # this is a comment alias_name=nic # some base string. current network drivers like rtl, wi would # be forbidden wi0=0 # numeric field, unique for each interface le0=1 00:40:a7:0b:13:70=2 = The configuration above would make your wi1 interface available as nic0, your le0 interface would be named nic1 and the interface that holds the mac address 00:40:a7:0b:13:70 would be visible as nic2 So, the following commands should be considered equal. ifconfig le0 ifconfig nic1 The feature described above would have huge collateral effects to lots of things and I can't say a patch would pass to mainstream. I also never did any research beyond this superficial layer. Just some early-morning thoughts, for what their worth. Doug.
Re: Packet Filter: how to keep device names on hardware failure?
Harald Dunkel [EMAIL PROTECTED] writes: Sorry to wake this thread up again, but this problem is a severe security risk. IMHO it is unacceptable that a hardware failure on one NIC of a firewall can put the whole network at risk, just because the mapping between NICs and interface names gets mixed up, and PF suddenly treats the Internet as a subnet of the company LAN. Semi-random reordering of network interfaces would be a severe problem, no doubt. However, my hazy memory was that reordering would not occur as you describe, but ICBW, please correct me if this has actually been demonstrated to happen. The workarounds posted here just show that OpenBSD's native functionality is insufficient. If you are planning a version 5.0, then it would be very nice if you could address this problem. Version 5.0 would be roughly three years into the future unless OpenBSD chooses to deviate from the version naming scheme, ie 4.5 - May 2009 4.6 - November 2009 4.7 - May 2010 4.8 - November 2010 4.9 - May 2011 5.0 - November 2011 any issues that have real security impact would likely be handled *a lot* sooner. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: Packet Filter: how to keep device names on hardware failure?
On 2008-11-07, Harald Dunkel [EMAIL PROTECTED] wrote: Peter N. M. Hansteen wrote: Harald Dunkel [EMAIL PROTECTED] writes: Sorry to wake this thread up again, but this problem is a severe security risk. IMHO it is unacceptable that a hardware failure on one NIC of a firewall can put the whole network at risk, just because the mapping between NICs and interface names gets mixed up, and PF suddenly treats the Internet as a subnet of the company LAN. Semi-random reordering of network interfaces would be a severe problem, no doubt. However, my hazy memory was that reordering would not occur as you describe, but ICBW, please correct me if this has actually been demonstrated to happen. I can post 2 dmesg logs of the same machine with the NIC names mixed up. Somehow 2 NICs disappeared on a reboot. On the next reboot they were back. Attached is the diff. This isn't entirely unknown with some quad em devices on some motherboards..
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, 7 Nov 2008, Harald Dunkel wrote: Peter N. M. Hansteen wrote: Harald Dunkel [EMAIL PROTECTED] writes: Sorry to wake this thread up again, but this problem is a severe security risk. IMHO it is unacceptable that a hardware failure on one NIC of a firewall can put the whole network at risk, just because the mapping between NICs and interface names gets mixed up, and PF suddenly treats the Internet as a subnet of the company LAN. Semi-random reordering of network interfaces would be a severe problem, no doubt. However, my hazy memory was that reordering would not occur as you describe, but ICBW, please correct me if this has actually been demonstrated to happen. I can post 2 dmesg logs of the same machine with the NIC names mixed up. Somehow 2 NICs disappeared on a reboot. On the next reboot they were back. Attached is the diff. In the bad configuration the NIC with 00:30:48:d2:9a:06 is called em2, in the good one it is called em4. Maybe you can imagine how PF screws up, if this NIC would have been physically connected to the Internet. Surely it is unusual that a NIC disappears somehow. Maybe there is something wrong with my hardware, but this can always happen. I would like to have a secure setup even if there is a hardware failure. Network configuration has bugged me a bit ever since I started using OpenBSD, not just the real security issue that Harald Dunkel points out but general ease of administration issues. For example, on a typical single-NIC system one ought to be able to set up a standard configuration and not care which make/model of NIC is installed. Perhaps most of these issues could be dealt with by changing the network configuration procedure to have a hierarchy of interface-configuration files rather than just hostname.interface-name. If hostname.mac were used if the hardware MAC matches, then hostname.interface-name, then (say) hostname.only if there's only one NIC found, the sysadmin could assign interfaces to groups and use those group names everywhere, and so not need to use the actual interface names at all. This appears to be a fairly simple change. Does it sound reasonable to people with more knowledge of OpenBSD networking? Dave -- Dave Anderson [EMAIL PROTECTED]
Re: Packet Filter: how to keep device names on hardware failure?
Peter N. M. Hansteen wrote: Harald Dunkel [EMAIL PROTECTED] writes: Sorry to wake this thread up again, but this problem is a severe security risk. IMHO it is unacceptable that a hardware failure on one NIC of a firewall can put the whole network at risk, just because the mapping between NICs and interface names gets mixed up, and PF suddenly treats the Internet as a subnet of the company LAN. Semi-random reordering of network interfaces would be a severe problem, no doubt. However, my hazy memory was that reordering would not occur as you describe, but ICBW, please correct me if this has actually been demonstrated to happen. I can post 2 dmesg logs of the same machine with the NIC names mixed up. Somehow 2 NICs disappeared on a reboot. On the next reboot they were back. Attached is the diff. In the bad configuration the NIC with 00:30:48:d2:9a:06 is called em2, in the good one it is called em4. Maybe you can imagine how PF screws up, if this NIC would have been physically connected to the Internet. Surely it is unusual that a NIC disappears somehow. Maybe there is something wrong with my hardware, but this can always happen. I would like to have a secure setup even if there is a hardware failure. Network configuration has bugged me a bit ever since I started using OpenBSD, not just the real security issue that Harald Dunkel points out but general ease of administration issues. For example, on a typical single-NIC system one ought to be able to set up a standard configuration and not care which make/model of NIC is installed. You are joking right? In that case you use the egress interface group. It works perfectly on all machines with one network interface, and follows the default route. Or would you rather use eth0? Perhaps most of these issues could be dealt with by changing the network configuration procedure to have a hierarchy of interface-configuration files rather than just hostname.interface-name. Oh, like eth0 and eth1 and eth2? If hostname.mac were used if the hardware MAC matches, then hostname.interface-name, then (say) hostname.only if there's only one NIC found, the sysadmin could assign interfaces to groups and use those group names everywhere, and so not need to use the actual interface names at all. So right now you have hardware that is disappearing. What happens when you get hardware where the MAC reading is not 100% reliable. New problem, and a new solution? This appears to be a fairly simple change. Does it sound reasonable to people with more knowledge of OpenBSD networking? No, it is not reasonble. You are inventing problems at a very high level just because some very low level pci-related bug is making some of your devices not reliably show themselves.
Re: Packet Filter: how to keep device names on hardware failure?
On Nov 7, 2008, at 9:44 AM, Dave Anderson wrote: Network configuration has bugged me a bit ever since I started using OpenBSD, not just the real security issue that Harald Dunkel points out but general ease of administration issues. For example, on a typical single-NIC system one ought to be able to set up a standard configuration and not care which make/model of NIC is installed. The HAL feature of Linux has always been more frustrating and annoying than helpful. When I have multiple interfaces in a Linux system, I sometimes discover that devices aren't always enumerated in the same order. Or vanish, or if I change out hardware it suddenly adds a new interface vs just replacing the old hardware. I have a single interface, why would I want there to be an eth1 when I swap out one EtherExpress card for another? I've had all of these problems with HAL in Linux. Perhaps most of these issues could be dealt with by changing the network configuration procedure to have a hierarchy of interface-configuration files rather than just hostname.interface-name. If hostname.mac were used if the hardware MAC matches, then hostname.interface-name, then (say) hostname.only if there's only one NIC found, the sysadmin could assign interfaces to groups and use those group names everywhere, and so not need to use the actual interface names at all. Why not standardize your hardware, instead? This appears to be a fairly simple change. Does it sound reasonable to people with more knowledge of OpenBSD networking? It's not a simple change.
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, Nov 7, 2008 at 3:51 AM, Harald Dunkel [EMAIL PROTECTED] wrote: Question: How can I make sure that em2 doesn't become em0 if my dual-port NIC dies? This would be fatal for my firewall setup. At least the antispoof rules _must_ be bound to the network devices. Sorry to wake this thread up again, but this problem is a severe security risk. IMHO it is unacceptable that a hardware failure on one NIC of a firewall can put the whole network at risk, just because the mapping between NICs and interface names gets mixed up, and PF suddenly treats the Internet as a subnet of the company LAN. echo 'if ! ifconfig | grep em2 /dev/null; then mail -s Tube explosion [EMAIL PROTECTED]; shutdown -h now; fi' /etc/rc.local
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, 07 Nov 2008 13:22:08 +0100, Peter N. M. Hansteen wrote: Harald Dunkel [EMAIL PROTECTED] writes: I can post 2 dmesg logs of the same machine with the NIC names mixed up. Somehow 2 NICs disappeared on a reboot. On the next reboot they were back. Attached is the diff. Dodgy hardware does lead to problems, certainly. The basic problem here is that the system enumerates only the hardware it is able to contact and identify, and units of the same type are assigned an identifier equal to driver name plus a sequence number. Unless we make some other unique identifier part of the way PF evaluates rules (the MAC address comes to mind, but that too can be changed in any modern operating system), there is no quick fix, other than rewriting your rule set so it avoids 'on' criteria and other hardware specifics wherever possible. Let's look at this a little more analytically: My firewall is a Soekris 4801 with sis0, sis1 and sis2. sis0 is the 0utside (ADSL) sis1 is the 1nside (LAN) sis2 is the 2erver LAN If 0 fails the other two move up the table. Risk = zero. If 1 fails the users holler No service! and the servers won't be compromised because they will now be connected to sis2 promoted to be sis1 and their default route won't be available and incoming traffic can't get to them either. Now, what was the problem again? With all the interfaces below the failure moving up the table there will be address mismatches = no traffic. I see no reason to panic. Maybe I'm too tired after being up really late replacing a faulty modem and I forgot to turn off NAT in the new one so my sleepy eyes missed the fact that I needed to test more than browsing from the LAN to make sure my servers were reachable. 8-(( 8 snip rest of story. *** NOTE *** Please DO NOT CC me. I am subscribed to the list. Mail to the sender address that does not originate at the list server is tarpitted. The reply-to: address is provided for those who feel compelled to reply off list. Thankyou. Rod/ /earth: write failed, file system is full cp: /earth/creatures: No space left on device
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, Nov 7, 2008 at 12:44 PM, Dave Anderson [EMAIL PROTECTED] wrote: Network configuration has bugged me a bit ever since I started using OpenBSD, not just the real security issue that Harald Dunkel points out but general ease of administration issues. For example, on a typical single-NIC system one ought to be able to set up a standard configuration and not care which make/model of NIC is installed. This is where pound enter until done in the installer just works.
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, 7 Nov 2008, Ted Unangst wrote: On Fri, Nov 7, 2008 at 12:44 PM, Dave Anderson [EMAIL PROTECTED] wrote: Network configuration has bugged me a bit ever since I started using OpenBSD, not just the real security issue that Harald Dunkel points out but general ease of administration issues. For example, on a typical single-NIC system one ought to be able to set up a standard configuration and not care which make/model of NIC is installed. This is where pound enter until done in the installer just works. True if you're installing; I _like_ the OpenBSD installation procedure. But it doesn't help if you replace a NIC in an already-built system with a different make/model or need to move one to a different slot. Do note that I consider the single-NIC ease-of-use issue to be a minor one; it just came up as fallout from thinking about the more-important unexpected-name-change issue. Dave -- Dave Anderson [EMAIL PROTECTED]
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, 7 Nov 2008, Theo de Raadt wrote: Peter N. M. Hansteen wrote: Harald Dunkel [EMAIL PROTECTED] writes: Sorry to wake this thread up again, but this problem is a severe security risk. IMHO it is unacceptable that a hardware failure on one NIC of a firewall can put the whole network at risk, just because the mapping between NICs and interface names gets mixed up, and PF suddenly treats the Internet as a subnet of the company LAN. Semi-random reordering of network interfaces would be a severe problem, no doubt. However, my hazy memory was that reordering would not occur as you describe, but ICBW, please correct me if this has actually been demonstrated to happen. I can post 2 dmesg logs of the same machine with the NIC names mixed up. Somehow 2 NICs disappeared on a reboot. On the next reboot they were back. Attached is the diff. In the bad configuration the NIC with 00:30:48:d2:9a:06 is called em2, in the good one it is called em4. Maybe you can imagine how PF screws up, if this NIC would have been physically connected to the Internet. Surely it is unusual that a NIC disappears somehow. Maybe there is something wrong with my hardware, but this can always happen. I would like to have a secure setup even if there is a hardware failure. Network configuration has bugged me a bit ever since I started using OpenBSD, not just the real security issue that Harald Dunkel points out but general ease of administration issues. For example, on a typical single-NIC system one ought to be able to set up a standard configuration and not care which make/model of NIC is installed. You are joking right? In that case you use the egress interface group. It works perfectly on all machines with one network interface, and follows the default route. Maybe I'm just confused, but my recollection is that one needs to set up the appropriate hostname.interface-name to enable the interface before the egress interface group works. This single-NIC case is admittedly a minor one, but it would eliminate one of the few (that I can think of) things about running a basic OpenBSD system that requires any arcane setup. Or would you rather use eth0? Since I never mentioned eth0, the answer is pretty obviously no. Perhaps most of these issues could be dealt with by changing the network configuration procedure to have a hierarchy of interface-configuration files rather than just hostname.interface-name. Oh, like eth0 and eth1 and eth2? See above. If hostname.mac were used if the hardware MAC matches, then hostname.interface-name, then (say) hostname.only if there's only one NIC found, the sysadmin could assign interfaces to groups and use those group names everywhere, and so not need to use the actual interface names at all. So right now you have hardware that is disappearing. What happens when you get hardware where the MAC reading is not 100% reliable. New problem, and a new solution? Actually, it's other people who have the problem. I was just speculating on how to minimize its harmful effects. One can always postulate a hardware (or other) failure which can't be dealt with by whatever the current software may be; the question is whether it happens often enough and is serious enough to be worth doing something about. Or if it suggests a change which is worthwhile in itself and also solves the problem. This appears to be a fairly simple change. Does it sound reasonable to people with more knowledge of OpenBSD networking? No, it is not reasonble. You are inventing problems at a very high level just because some very low level pci-related bug is making some of your devices not reliably show themselves. No, I'm thinking about a general way for those people who care about it to tie pf rules, etc, to specific physical interfaces, regardless of what other devices are installed or configured in a system. Dave -- Dave Anderson [EMAIL PROTECTED]
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, Nov 7, 2008 at 3:55 PM, Dave Anderson [EMAIL PROTECTED] wrote: Maybe I'm just confused, but my recollection is that one needs to set up the appropriate hostname.interface-name to enable the interface before the egress interface group works. ... haven't tried this, but maybe you can use the add command in ukc / config to create the constant device mappings you expect... maybe. -- GDB has a 'break' feature; why doesn't it have 'fix' too?
Re: Packet Filter: how to keep device names on hardware failure?
One can always postulate a hardware (or other) failure which can't be dealt with by whatever the current software may be; the question is whether it happens often enough and is serious enough to be worth doing something about. Or if it suggests a change which is worthwhile in itself and also solves the problem. It rarely happens. It cannot be worked around. Layering piles of workarounds is never the right path.
Re: Packet Filter: how to keep device names on hardware failure?
I see no problem in setting interface groups based on mac address. You should be able to hack a suitable script to do that in a few minutes.
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, 7 Nov 2008, Chris Kuethe wrote: On Fri, Nov 7, 2008 at 3:55 PM, Dave Anderson [EMAIL PROTECTED] wrote: Maybe I'm just confused, but my recollection is that one needs to set up the appropriate hostname.interface-name to enable the interface before the egress interface group works. ... haven't tried this, but maybe you can use the add command in ukc / config to create the constant device mappings you expect... maybe. I'm not expecting anything, just thinking about how to better handle those cases where it's important that pf rules, etc, operate on a specific physical interface (regardless of what other devices are installed or configured in the system). I've never yet had occasion to tinker with config/ukc, but in looking at its manpage and experimenting with it a bit I don't see any obvious way of specifying a particular physical device regardless of what slot it's in -- so I don't think this could accomplish what I'm looking to do. What's needed is a unique identifier for each physical device and a way to key off of it; for ethernet and fiber NICs, at least, the hardware MAC address is the only obvious candidate for such an ID. Dave -- Dave Anderson [EMAIL PROTECTED]
Re: Packet Filter: how to keep device names on hardware failure?
Hi Henning, I had posted my startup script configuring network interfaces according to their MAC address here. But it is still not a big help, since brconfig doesn't accept the interface group names. :-( Regards Harri === Henning Brauer wrote: * Harald Dunkel [EMAIL PROTECTED] [2008-08-22 16:33]: Question: How can I make sure that em2 doesn't become em0 if my dual-port NIC dies? [EMAIL PROTECTED] $ dmesg | grep '^em0' em0 at pci5 dev 0 function 0 Intel PRO/1000 PT (80003ES2) rev 0x01: apic 2 int 18 (irq 11), address 00:18:f3:70:b1:74 then you are a bit doomed, as config(8) is not enough to change your kernel acording, thus you need to build your own. [EMAIL PROTECTED] $ cd /usr/src/sys/arch/`arch -s`/conf/ [EMAIL PROTECTED] conf $ cat TEST EOF include arch/i386/conf/GENERIC em0 at pci5 dev0 function 0 EOF [EMAIL PROTECTED] conf $ config TEST TEST:2: em0 at pci5 is orphaned (no pci5 declared) *** Stop. too bad. now you need to declare pci5 statically (get from your dmesg), and everything that leads to it (ppbX, maybe a pciX that ppbX sits on, I leave that to you). worth it? dunno. a script at boot that checks mac addresses and makes creative use of interface groups might be easier.
Re: Packet Filter: how to keep device names on hardware failure?
Hi Jared, jared r r spiegel wrote: On Fri, Aug 22, 2008 at 04:16:38PM +0200, Harald Dunkel wrote: Hi folks, Question: How can I make sure that em2 doesn't become em0 if my dual-port NIC dies? This would be fatal for my firewall setup. At least the antispoof rules _must_ be bound to the network devices. first thing that comes to mind is to create unique interface groups for each iface and then write pf based on that. Thats a cool idea. I did not really recognize the group names before. I could parse the output of ifconfig while the Packet Filter is blocking everything, assign new interface group names according to the MAC address, and finally load a new pf.conf using group names. Many thanx Harri
Re: Packet Filter: how to keep device names on hardware failure?
PS: Below is the code, if anybody is interested. Should be run
before /etc/netstart. To use it you should create a file
/etc/ifconfig.xx:xx:xx:xx:xx:xx
for each network device (xx:xx:xx:xx:xx:xx is the MAC
address). Each line is run with
ifconfig if $line
Here is a sample
inet 172.19.96.1/24
inet alias 172.19.97.1/24
group int_if
Comments are welcome.
Regards
Harri
==
# /etc/netstart.local
#
# local netstart extension
#
echo 'create interface groups'
for i in $(ifconfig | grep '^[a-zA-Z0-9]*: flags=' | cut -d: -f1); do
mac=$(ifconfig $i | grep 'lladdr' | awk '{ print $2 }')
test $mac || continue
test -f /etc/ifconfig.$mac || continue
cat /etc/ifconfig.$mac | egrep -v '^[[:space:]]*#|^[[:space:]]*$' | while
read line; do
echo ifconfig $i $line
ifconfig $i $line
done
done
Re: Packet Filter: how to keep device names on hardware failure?
* Harald Dunkel [EMAIL PROTECTED] [2008-08-22 16:33]: Question: How can I make sure that em2 doesn't become em0 if my dual-port NIC dies? [EMAIL PROTECTED] $ dmesg | grep '^em0' em0 at pci5 dev 0 function 0 Intel PRO/1000 PT (80003ES2) rev 0x01: apic 2 int 18 (irq 11), address 00:18:f3:70:b1:74 then you are a bit doomed, as config(8) is not enough to change your kernel acording, thus you need to build your own. [EMAIL PROTECTED] $ cd /usr/src/sys/arch/`arch -s`/conf/ [EMAIL PROTECTED] conf $ cat TEST EOF include arch/i386/conf/GENERIC em0 at pci5 dev0 function 0 EOF [EMAIL PROTECTED] conf $ config TEST TEST:2: em0 at pci5 is orphaned (no pci5 declared) *** Stop. too bad. now you need to declare pci5 statically (get from your dmesg), and everything that leads to it (ppbX, maybe a pciX that ppbX sits on, I leave that to you). worth it? dunno. a script at boot that checks mac addresses and makes creative use of interface groups might be easier. -- Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED] BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg Amsterdam
Re: Packet Filter: how to keep device names on hardware failure?
On Fri, Aug 22, 2008 at 04:16:38PM +0200, Harald Dunkel wrote: Hi folks, Question: How can I make sure that em2 doesn't become em0 if my dual-port NIC dies? This would be fatal for my firewall setup. At least the antispoof rules _must_ be bound to the network devices. first thing that comes to mind is to create unique interface groups for each iface and then write pf based on that. you'll still have to deal with the fallout after reboot after a failure, but at least if the hardware for whatever reason did happen to disappear during operation, you'd be insulated against the immediate change (tho maybe pf already handles that) other than that, assuming the PCI locations or whatever stay consistent through reboots (like, put 3 nics in, boot, see where they are, pull the middle one, see if 1 and 3 are still at the same points in dmesg even tho their ifnums will change), you could maybe break apart the 'em* at pci*' (or whatever it is) in config(8) and make individual ones based on where you want them. if that doesn't work in config(8) you probably have to make your own kernel. so you could do a little work and get a marginal benefit or spend a (potentially *LOT*) lot more time and force things specifically. barring any better suggestions, of course. -- jared
Re: Packet Filter: how to keep device names on hardware failure?
Question: How can I make sure that em2 doesn't become em0 if my dual-port NIC dies? This would be fatal for my firewall setup. At least the antispoof rules _must_ be bound to the network devices. Yep, this is an ugly problem. You could have a shellscript at boot scan ifconfig output and associate NICs with their MAC addresses, adding appropriate macros to pf.conf.
