Re: 3.8 beta requests / test result on HP DL360
On 23 Aug 2005, at 01:33, Theo de Raadt wrote: We are heading towards making the real 3.8 release soonish. I would like to ask the community to do lots of testing over the next week if they can. For info, here is the latest 3.8 i386 snapshot booting on a 'common corporate workhorse' HP DL360, w/ 3GB RAM 2xCPU and single RAID1 logical disk. Notes: 1. micky's new ciss raid driver work very well, although spits a few ciss0: cmd_stat 2 scsi_stat 0x0 from time to time. 2. The second NIC (bge1) fails to be attached on a single processor kernel. Anyone got any suggestions for BIOS/boot tweaks to get this working ? 3. if bsd.mp is booted then it drops into ddb trying to attach bge1. I can try for a com port ps trace if requested. 4. as mentioned in theo's mail 09-09-2005, bioctl supports only ami so far - I wonder if ciss support is likely ? a documentation issue I suspect. 5. dmesg below: OpenBSD 3.8 (GENERIC) #137: Thu Sep 1 17:41:20 MDT 2005 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC cpu0: Intel(R) Xeon(TM) CPU 2.80GHz (GenuineIntel 686-class) 2.80 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36, CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID real mem = 3220783104 (3145296K) avail mem = 2931396608 (2862692K) using 4278 buffers containing 161140736 bytes (157364K) of memory mainbus0 (root) bios0 at mainbus0: AT/286+(00) BIOS, date 12/31/99, BIOS32 rev. 0 @ 0xf pcibios0 at bios0: rev 2.1 @ 0xf/0x2000 pcibios0: PCI BIOS has 7 Interrupt Routing table entries pcibios0: PCI Interrupt Router at 000:15:0 (ServerWorks CSB5 SouthBridge rev 0x00) pcibios0: PCI bus #0 is the last bus bios0: ROM list: 0xc/0x8000 0xc8000/0x4000 0xee000/0x2000! cpu0 at mainbus0 pci0 at mainbus0 bus 0: configuration mode 1 (no bios) pchb0 at pci0 dev 0 function 0 ServerWorks CNB20-HE rev 0x31 pchb1 at pci0 dev 0 function 1 ServerWorks CNB20-HE rev 0x00 pchb2 at pci0 dev 0 function 2 ServerWorks CNB20-HE rev 0x00 pci1 at pchb2 bus 1 bge0 at pci1 dev 2 function 0 Broadcom BCM5703X rev 0x02, BCM5703 A2 (0x1002): irq 11 address 00:0b:cd:4e:4a:3a brgphy0 at bge0 phy 1: BCM5703 10/100/1000baseT PHY, rev. 2 vga1 at pci0 dev 3 function 0 ATI Rage XL rev 0x27 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) ciss0 at pci0 dev 4 function 0 Compaq Smart Array 5i/532 rev.2 rev 0x01: irq 3 ciss0: 1 LD HW rev 1 FW 2.36/2.36 lmap 4000:0 scsibus0 at ciss0: 1 targets sd0 at scsibus0 targ 0 lun 0: COMPAQ, LOGICAL VOLUME, 2.36 SCSI0 0/direct fixed ciss0: cmd_stat 2 scsi_stat 0x0 ciss0: cmd_stat 2 scsi_stat 0x0 sd0: 34727MB, 34727 cyl, 64 head, 32 sec, 512 bytes/sec, 71122560 sec total vendor Compaq, unknown product 0xb203 (class system subclass miscellaneous, rev 0x01) at pci0 dev 5 function 0 not configured vendor Compaq, unknown product 0xb204 (class system subclass miscellaneous, rev 0x01) at pci0 dev 5 function 2 not configured pcib0 at pci0 dev 15 function 0 ServerWorks CSB5 SouthBridge rev 0x93 pciide0 at pci0 dev 15 function 1 ServerWorks CSB5 IDE rev 0x93: DMA atapiscsi0 at pciide0 channel 0 drive 0 scsibus1 at atapiscsi0: 2 targets cd0 at scsibus1 targ 0 lun 0: COMPAQ, CD-ROM SN-124, N104 SCSI0 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4 ohci0 at pci0 dev 15 function 2 ServerWorks OSB4/CSB5 USB rev 0x05: irq 10, version 1.0, legacy support usb0 at ohci0: USB revision 1.0 uhub0 at usb0 uhub0: ServerWorks OHCI root hub, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered pchb3 at pci0 dev 15 function 3 ServerWorks CSB5 PCI rev 0x00 pchb4 at pci0 dev 17 function 0 ServerWorks CIOBX2 rev 0x05 pchb5 at pci0 dev 17 function 2 ServerWorks CIOBX2 rev 0x05 pci2 at pchb5 bus 4 bge1 at pci2 dev 2 function 0 Broadcom BCM5703X rev 0x02: couldn't establish interrupt at irq 15 isa0 at pcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pms0 at pckbc0 (aux slot) pckbc0: using irq 12 for aux slot wsmouse0 at pms0 mux 0 pcppi0 at isa0 port 0x61 midi0 at pcppi0: PC speaker spkr0 at pcppi0 sysbeep0 at pcppi0 npx0 at isa0 port 0xf0/16: using exception 16 pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec biomask e7ed netmask efed ttymask ffef pctr: user-level cycle counter enabled ciss0: cmd_stat 2 scsi_stat 0x0 ciss0: cmd_stat 2 scsi_stat 0x0 dkcsum: sd0 matches BIOS drive 0x80 root on sd0a ciss0: cmd_stat 2 scsi_stat 0x0 ciss0: cmd_stat 2 scsi_stat 0x0 ciss0: cmd_stat 2 scsi_stat 0x0 ciss0: cmd_stat 2 scsi_stat 0x0 rootdev=0x400 rrootdev=0xd00 rawdev=0xd02 ciss0: cmd_stat 2 scsi_stat 0x0 ciss0: cmd_stat 2 scsi_stat 0x0 ciss0: cmd_stat 2 scsi_stat 0x0 ciss0: cmd_stat 2 scsi_stat 0x0 # #
Re: 3.8 beta requests
Kevin [EMAIL PROTECTED] writes: Coming back to an open terminal that I know I locked was a bit of a shock. Almost makes me wonder if xlock can be trusted. If you use any mode other than a blank screen it's definitely not to be trusted. All those eye candy modes are designed to be eye candy and not stable and secure. //art
Re: 3.8 beta requests
On Wed, Aug 31, 2005 at 04:17:06PM -0500, Kevin wrote: On 8/31/05, Christopher Linn [EMAIL PROTECTED] wrote: On Wed, Aug 31, 2005 at 11:12:07AM -0600, Peter Valchev wrote: I've been testing 3.8 on a couple of i386 systems (soon sparc also), including installing more of the 3.8 beta packages than I would use normally. So far I am impressed by UP/MP performance, and have only found a couple of X applications (xtacy, xlock) failing on signal 11. ^ , that's a biggie.. I can't tell if you were being serious or sarcastic? at the risk of posting public reply to private email (i think it's probably OK in this case...) quite serious actually. my desktop is an openbsd box, and my office is adjacent to a semi-public area at a university. i frequently leave my office for a few minutes at a time up to a few hours. so, yes i really do depend on it. the ports@ mailing list is the best as well as the maintainers at this point. please don't wait, but let us know of the details so this can be fixed!! Will do. KK chris -- Christopher Linn celinn at mtu.edu | By no means shall either the CEC System Administrator II | or MTU be held in any way liable Center for Experimental Computation | for any opinions or conjecture I Michigan Technological University | hold to or imply to hold herein.
Re: 3.8 beta requests
On 9/1/05, Christopher Linn [EMAIL PROTECTED] wrote: On Wed, Aug 31, 2005 at 04:17:06PM -0500, Kevin wrote: On 8/31/05, Christopher Linn [EMAIL PROTECTED] wrote: Kevin Kadow wrote: only found a couple of X applications (xtacy, xlock) failing on signal 11. ^ , that's a biggie.. I can't tell if you were being serious or sarcastic? at the risk of posting public reply to private email (i think it's probably OK in this case...) It's OK. Glad that you thought twice about it, so many people don't. quite serious actually. my desktop is an openbsd box, and my office is adjacent to a semi-public area at a university. i frequently leave my office for a few minutes at a time up to a few hours. so, yes i really do depend on it. Coming back to an open terminal that I know I locked was a bit of a shock. Almost makes me wonder if xlock can be trusted. You'd think it'd set a sighandler when going into a mode, and not 'exit' just because one mode subroutine threw an exception. I'll send a diff to add an additional **WARNING** section to the man page. KK
Re: 3.8 beta requests
I've been testing 3.8 on a couple of i386 systems (soon sparc also), including installing more of the 3.8 beta packages than I would use normally. So far I am impressed by UP/MP performance, and have only found a couple of X applications (xtacy, xlock) failing on signal 11. the ports@ mailing list is the best as well as the maintainers at this point. please don't wait, but let us know of the details so this can be fixed!!
Re: 3.8 beta requests
On Wed, Aug 31, 2005 at 11:12:07AM -0600, Peter Valchev wrote: I've been testing 3.8 on a couple of i386 systems (soon sparc also), including installing more of the 3.8 beta packages than I would use normally. So far I am impressed by UP/MP performance, and have only found a couple of X applications (xtacy, xlock) failing on signal 11. ^ , that's a biggie.. the ports@ mailing list is the best as well as the maintainers at this point. please don't wait, but let us know of the details so this can be fixed!! chris -- Christopher Linn celinn at mtu.edu | By no means shall either the CEC System Administrator II | or MTU be held in any way liable Center for Experimental Computation | for any opinions or conjecture I Michigan Technological University | hold to or imply to hold herein.
Re: 3.8 beta requests
On 8/22/05, Theo de Raadt [EMAIL PROTECTED] wrote: We are heading towards making the real 3.8 release soonish. I would like to ask the community to do lots of testing over the next week if they can. What is the best way to test? Should we be downloading snapshots daily? Install snapshots. Install snapshot packages. Try using it as if it is the real 3.8. Tell us if things fail. By tell us, should we be contacting the port maintainer directly, using sendbug or both? I've been testing 3.8 on a couple of i386 systems (soon sparc also), including installing more of the 3.8 beta packages than I would use normally. So far I am impressed by UP/MP performance, and have only found a couple of X applications (xtacy, xlock) failing on signal 11. Kevin Kadow
Re: 3.8 beta requests
On Thu, 25 Aug 2005 14:57:37 +1000, Shane J Pearson wrote: Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ? However Genadijus only asked questions. He did not make a statement. Seems like pretty innocent questions to me that are easily answered here by those that know. And what is wrong with that? Thanks for taking the side of the poor and innocent, who too often are run aground in an arrogant manner in here. This is what I appreciate ! If you look at the specific question, though, it is more on the dumb to silly side. Simply going to www.openbsd.org will show that OpenBSD tries as hard or even harder than everyone else to ship a stable production-quality system. The FAQ will point out clearly that there are various flavours and Theo wrote in the OP that it has been running on intermediate version for quite some time. I do not like spontaneous questions being posed by people for whom googling for a few minutes or reading the project site for a similar amount of time is already too much work. My personal opinion, Uwe
Re: 3.8 beta requests
On Thu, 25 Aug 2005 14:34:35 +0800, Uwe Dippel wrote: whatever. Wrong post, wrong place. Discard ! Uwe
OpenBSD Wikipedia (was Re: 3.8 beta requests)
I have made breif changes to the OpenBSD page on wikipedia detailing the systems security regarding these new changes. My information may be slightly inaccurate or misleading, please feel free to check it. Diff here: http://en.wikipedia.org/w/index.php?title=OpenBSDdiff=21793744oldid=21739418 And then I made these changes to a big mistake regarding the actually version I mentioned, as you will see above. Diff here: http://en.wikipedia.org/w/index.php?title=OpenBSDdiff=21793991oldid=21793744 Yours, John. On 8/23/05, Theo de Raadt [EMAIL PROTECTED] wrote: We are heading towards making the real 3.8 release soonish. I would like to ask the community to do lots of testing over the next week if they can. This release will bring a lot of new ideas from us. One of them in particular is somewhat risky. I think it is time to talk about that one, and let people know what is ahead on our road. Traditionally, Unix malloc(3) has always just extended the brk, which means extending the traditional Unix process data segment to allocate more memory. malloc(3) would simply extend the data segment, and then calve off little pieces to requesting callers as needed. It also remembered which pieces were which, so that free(3) could do it's job. The way this was always done in Unix has had a number of consequences, some of which we wanted to get rid of. In particular, malloc free have not been able to provide strong protection against overflows or other corruption. Our malloc implementation is a lot more resistant (than Linux) to heap overflows in the malloc arena, but we wanted to improve things even more. Starting a few months ago, the following changes were made: - We made the mmap(2) system call return random memory addresses. As well the kernel ensures that two objects are not mapped next to each other; in effect, this creates unallocated memory which we call a guard page. - We have changed malloc(3) to use mmap(2) instead of extending the data segment via brk() - We also changed free(3) to return memory to the kernel, un-allocating them out of the process. - As before, objects smaller than a page are allocated within shared pages that malloc(3) maintains. But their allocation is now somewhat randomized as well. - A number of other similar changes which are too dangerous for normal software or cause too much of a slowdown are available as malloc options as described in the manual page. These are very powerful for debugging buggy applications. Other results: - When you free an object that is = 1 page in size, it is actually returned to the system. Attempting to read or write to it after you free is no longer acceptable. That memory is unmapped. You get a SIGSEGV. - For a decade and a bit, we have been fixing software for buffer overflows. Now we are finding a lot of software that reads before the start of the buffer, or reads too far off the end of the buffer. You get a SIGSEGV. To some of you, this will sound like what the Electric Fence toolkit used to be for. But these features are enabled by default. Electric Fence was also very slow. It took nearly 3 years to write these OpenBSD changes since performance was a serious consideration. (Early versions caused a nearly 50% slowdown). Our changes have tremendous benefits, but until some bugs in external packages are found and fixed, there are some risks as well. Some software making incorrect assumptions will be running into these new security technologies. I discussed this in talks I have given before: I said that we were afraid to go ahead with guard pages, because a lot of software is just written to such low standards. Applications over-read memory all the time, go 1 byte too far, read 1 byte too early, access memory after free, etc etc etc. Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Two examples: Over the last two months, some OpenBSD users noticed that the X server was crashing occasionally. Two bugs have been diagnosed and fixed by us. One was a use-after-free bug in the X shared library linker. The other was a buffer-over-read bug deep down in the very lowest level fb* pixmap compositing routines. The latter bug in particular was very difficult to diagnose and fix, and is about 10 years old. We have found other bugs like this in other external software, and even a few in the base OpenBSD tree (though those were found a while back, even as we started experimenting with the new malloc code). I would bet money that the X fb* bug has crashed Linux (and other) X servers before. It is just that it was very rare, and noone ever chased it. The new malloc we have just makes code get lucky less often, which lets us get to the source of a bug easier. As a programmer, I appreciate anything which makes bugs
Re: 3.8 beta requests
On 8/24/05, Ted Unangst [EMAIL PROTECTED] wrote: On Wed, 24 Aug 2005, Siju George wrote: just one quick question. where do I actually learn more about page, buffer, malloc etc?? Is this book enough? http://www.amazon.com/exec/obidos/ISBN%3D0201549794/openbsdA/104-8401808-3342305 or are there other good books out there? it's useful. the dinosaur book, _operating system concepts_, is also recommended. thanks a lot tedu :-) kind regards Siju -- And that's why I stopped reading the big newspapers.
Re: 3.8 beta requests
Thanks for not taking the easy route. Changes are always painful, but if they deliver then it's worth it.
Re: 3.8 beta requests
Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ?
Re: 3.8 beta requests
No,it is clear that he is talking about the problems *other* people's (buggy) software will have. On 8/24/05, Genadijus Paleckis [EMAIL PROTECTED] wrote: Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ?
Re: 3.8 beta requests
Antonios Anastasiadis wrote: No,it is clear that he is talking about the problems *other* people's (buggy) software will have. On 8/24/05, Genadijus Paleckis [EMAIL PROTECTED] wrote: Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ? well, from base system side I gues it will be minimal problems, but what about ports ? because almost everyone using it.
Re: 3.8 beta requests
Genadijus Paleckis wrote: Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ? Maybe, maybe not. Perhaps you like worrying? Anyway. I've been testing this stuff since the first snapshots and now the 3.8 beta and I never noticed any instability. # Han -- . When a place gets crowded enough to require ID's, social ..^/ collapse is not far away. It is time to go elsewhere. The `-. ___ ) best thing about space travel is that it made it possible to || || mh go elsewhere. -- Robert Heinlein, Time Enough For Love
Re: 3.8 beta requests
Genadijus Paleckis [EMAIL PROTECTED] writes: Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ? Yes, it means you should switch to linux because it's stable and never does anything to rock the boat. sigh. It's comments like this that convince me that I should never tell anyone about what I'm developing, how it works and what effects it might have. Anything you say will be used against you. //art
Re: 3.8 beta requests
Theo de Raadt wrote: Of course not. HOW CAN IT? Get real! The hardware is STILL only providing permissions at the page level! If you have aggressive amounts of ram and/or patience you could have something along the malloc.conf P-option for ALL sizes. Of course it would suck for any app more complex than sleep but for the sake of argument... Apparently the new malloc(3) implementation doesn't stop me from writing past the end of buffer as long as I am inside the last page. (Please forgive me beforehand if I am missing something too obvious)
Re: 3.8 beta requests
Hello! On Wed, Aug 24, 2005 at 02:28:25PM +0300, Genadijus Paleckis wrote: [...] Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ? well, from base system side I gues it will be minimal problems, but what about ports ? because almost everyone using it. The very most things just work for me. Base, X11, applications like firefox or gaim, own C/C++ code. A few things that get bitten are some packages doing their own and very different memory management, but can't avoid malloc altogether. That is ports/lang/clisp, that seems to be also gprolog, according to Marc Espie. I'd guess it'll also bite sbcl/cmucl (but there's no current port [neither in the sense of /usr/ports, nor in the sense of a 3rd party package] of cmucl for OpenBSD anyway). Some other things are not bitten in the same way, even though they do have different memory management. Including ghc, probably also SML/NJ (own build as of Jul 12, using libc 38.1, wasn't mmap-based malloc + mmap randomization in there already?). I *am* a bit sad about the fact that there're no running Lisp implementations for OpenBSD at all in the moment, but I don't have the energy to contribute own effort to change this, and it's not *that* high priority for me. I think Theo's (and other core developers') decision to release 3.8 with those malloc/mmap changes in is good overall. Kind regards, Hannah.
Re: 3.8 beta requests
On 2005/08/24 14:28:25, Genadijus Paleckis wrote: well, from base system side I gues it will be minimal problems, but what about ports ? because almost everyone using it. If software segfaults because of this, it's because it's already doing something wrong, and it could already be giving unpredictable results. If software is faulty, I'd rather have a segfault when the faulty code is run, than through finding corrupt data maybe months in the future because the failure was invisible.
Re: 3.8 beta requests
Artur Grabowski wrote: Genadijus Paleckis [EMAIL PROTECTED] writes: Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ? It's comments like this that convince me that I should never tell anyone about what I'm developing, how it works and what effects it might have. Anything you say will be used against you. Ow come on. What a one sided comment :-) Lots of people read it and rejoice. And lots of people dedicate a non-critical machine to running snapshots and try to find bugs. And I haven't found any malloc related problems since 3.7 :-) # Han -- OpenBSD: Only one remote ,`o. Consultants are mystical people who hole in the default install ( ,c@ ask a company for a number and then in more than 8 years!',,,' give it back to them.
Re: 3.8 beta requests
Genadijus Paleckis wrote: Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ? It means that you should try it and report bugs if you find any. Remember that most of the developers run -current throughout the development cycle (often in production). -d
Re: 3.8 beta requests
Hello! On Wed, Aug 24, 2005 at 08:02:54AM -0500, Dave Feustel wrote: On Wednesday 24 August 2005 07:04, Hannah Schroeter wrote: I *am* a bit sad about the fact that there're no running Lisp implementations for OpenBSD Does (X)emacs work? Yes, but I meant (and neglected to say explicitly) Common Lisp. Kind regards, Hannah.
Re: 3.8 beta requests
On Wednesday 24 August 2005 07:04, Hannah Schroeter wrote: A few things that get bitten are some packages doing their own and very different memory management, but can't avoid malloc altogether. That is ports/lang/clisp, that seems to be also gprolog Can you describe how these programs manage to seg fault doing their memory management? How do they run now if they don't use malloc? -- Tired of having to defend against Malware? (You know: trojans, viruses, SPYWARE, ADWARE, KEYLOGGERS, rootkits, worms and popups) Then Switch to OpenBSD with a KDE desktop!!!
Re: 3.8 beta requests
On Wednesday 24 August 2005 08:04, Hannah Schroeter wrote: Hello! On Wed, Aug 24, 2005 at 08:02:54AM -0500, Dave Feustel wrote: On Wednesday 24 August 2005 07:04, Hannah Schroeter wrote: I *am* a bit sad about the fact that there're no running Lisp implementations for OpenBSD Does (X)emacs work? Yes, but I meant (and neglected to say explicitly) Common Lisp. I understood what you meant. I was just wondering if everything using lisp techniques (eg scheme) was broken. Thanks. Kind regards, Hannah. -- Tired of having to defend against Malware? (You know: trojans, viruses, SPYWARE, ADWARE, KEYLOGGERS, rootkits, worms and popups) Then Switch to OpenBSD with a KDE desktop!!!
Re: 3.8 beta requests
On Wed, 24 Aug 2005, Damien Miller wrote: Remember that most of the developers run -current throughout the development cycle (often in production). -d and Theo get's really pissed off when someone breaks the tree so it won't compile and/or the change creates disfunction in other parts of the system, just read some of Theo's comments in the CVS list sometime. g.day
Re: 3.8 beta requests
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Diana Eichert Sent: Wednesday, August 24, 2005 10:08 AM To: Miscellaneous OBSD Subject: Re: 3.8 beta requests On Wed, 24 Aug 2005, Damien Miller wrote: Remember that most of the developers run -current throughout the development cycle (often in production). -d and Theo get's really pissed off when someone breaks the tree so it won't compile and/or the change creates disfunction in other parts of the system, just read some of Theo's comments in the CVS list sometime. g.day In the end, quality control happens through selfish testing. The OpenBSD community doesn't evenly divide up the things to test. People test their own setups. I'm not concerned with making OpenBSD stable. I'm concerned with making i386 OpenBSD running Mambo stable. The wonderful thing about a participatory development process is that everyone's overlapping needs generally test the system fairly well. The real problem is people who encounter a problem and fail to report it. They just think this is crap and go on to something else.
Re: 3.8 beta requests
hmm, on Tue, Aug 23, 2005 at 09:23:27AM -0700, Raymond Lillard said that Maybe a slogan along the lines of, Is your software good enough for OpenBSD!! Perhaps it could be worked into the release's theme. that is truly a brilliant idea ;-) any artists here? make a designed for puffy logo. first, all of the openbsd related projects could put it on their site. later the porters could ask their ported projects to include the logo on their page (if they deserve it) tshirts, mugs, a magazine, a tv show, finally even the HW manufacturers and microsoft would be pressed to redesign their OS to get the seal of quality. and after the planet is conquered, the universe is the limit! ha ha ha! -f (ps. i swear the tagline was generated random!) -- all your base are belong to us.
Re: 3.8 beta requests
On Wed, Aug 24, 2005 at 08:09:36AM -0500, Dave Feustel wrote: On Wednesday 24 August 2005 07:04, Hannah Schroeter wrote: A few things that get bitten are some packages doing their own and very different memory management, but can't avoid malloc altogether. That is ports/lang/clisp, that seems to be also gprolog Can you describe how these programs manage to seg fault doing their memory management? How do they run now if they don't use malloc? -- Those programs use mmap() to create their basic image and fill it in. Then on a later invocation, they try to use mmap() again to get the image at the same location, which works on most Unix systems, except for OpenBSD-current...
Re: 3.8 beta requests
On 8/25/05, -f [EMAIL PROTECTED] wrote: hmm, on Tue, Aug 23, 2005 at 09:23:27AM -0700, Raymond Lillard said that Maybe a slogan along the lines of, Is your software good enough for OpenBSD!! Perhaps it could be worked into the release's theme. that is truly a brilliant idea ;-) any artists here? make a designed for puffy logo. first, all of the openbsd related projects could put it on their site. later the porters could ask their ported projects to include the logo on their page (if they deserve it) How about we go Torvalds style and sue motherfuckers for trademark violations if they use it when they don't deserve it. tshirts, mugs, a magazine, a tv show, finally even the HW manufacturers and microsoft would be pressed to redesign their OS to get the seal of quality. and after the planet is conquered, the universe is the limit! ha ha ha! -f (ps. i swear the tagline was generated random!) -- all your base are belong to us. -- John Kintaro Tate Mobile: 0413 348 815 (Yep, old number, but I have a new phone) Attention all Internet users, is life getting you down? Are you so happy you could chainsaw an innocent bystander and LAUGH? Do you believe in God? Do you not believe in God? Have you found yourself stranded on prehistoric Earth for 5 years? If so, if you do anything at all there are people who care at the Kintaro Labs Forum, join now and after you reach 50 posts you get a free OpenBSD shell account! http://labs.kintaro.noobify.com Personal Website: http://kintaro.noobify.com
Re: 3.8 beta requests
On Wednesday 24 August 2005 10:56, Marc Espie wrote: On Wed, Aug 24, 2005 at 08:09:36AM -0500, Dave Feustel wrote: On Wednesday 24 August 2005 07:04, Hannah Schroeter wrote: A few things that get bitten are some packages doing their own and very different memory management, but can't avoid malloc altogether. That is ports/lang/clisp, that seems to be also gprolog Can you describe how these programs manage to seg fault doing their memory management? How do they run now if they don't use malloc? -- Those programs use mmap() to create their basic image and fill it in. Then on a later invocation, they try to use mmap() again to get the image at the same location, which works on most Unix systems, except for OpenBSD-current... In other words, now in OpenBSD 3.8, all addresses within an mmap'd region have to be treated as relative to the base address of the region if the region is mapped more than once? -- Tired of having to defend against Malware? (You know: trojans, viruses, SPYWARE, ADWARE, KEYLOGGERS, rootkits, worms and popups) Then Switch to OpenBSD with a KDE desktop!!!
Re: 3.8 beta requests
A few things that get bitten are some packages doing their own and very different memory management, but can't avoid malloc altogether. That is ports/lang/clisp, that seems to be also gprolog Can you describe how these programs manage to seg fault doing their memory management? How do they run now if they don't use malloc? Most of those that fail assume that if malloc returns a predictable memory address sequence. Not even emacs does that (and you don't want to hear that rant :)
Re: 3.8 beta requests
The real problem is people who encounter a problem and fail to report it. They just think this is crap and go on to something else. I think the developers need to address the problems that get brought up, too. I took the time to post a complete bug report (good and failing dmesg) about a bug that made an(4) crash the kernel and not boot 3.7 to misc@ and bugs@, then later sent it to the maintainer (mickey) , and got nothing each time, not even a yeah, okay we got it or take a look in this part of the code or try this message. It was very frustrating to try and make things better and get ignored. -- Hardware, n.: The parts of a computer system that can be kicked.
Re: 3.8 beta requests
Hello! On Wed, Aug 24, 2005 at 12:57:27PM -0500, Andrew Dyer wrote: It was very frustrating to try and make things better and get ignored. I can share some frustration. About a year ago, I made a port for erlang (the current port just doesn't work at all, and it's ancient anyway, so *anything* is better than the in-tree port). IIRC got feedback by one other person that it basically works. Nothing got committed, I didn't have the energy to follow on upon it. A few months later, someone asked about erlang, I answered and mailed the port of last summer, then IIRC that someone made an updated port (a newer Erlang release was out, and a few changes in the ports infrastructure) and submitted it. Again, nothing got committed, even though just *anything* would be better than the in-tree port. Kind regards, Hannah.
Re: 3.8 beta requests
Hi Art, On 24/08/2005, at 9:38 PM, Artur Grabowski wrote: Genadijus Paleckis [EMAIL PROTECTED] writes: Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Is that means that 3.8 might be unstable ? Maybe all who wants/needs stable systems need to run 3.7 ? Yes, it means you should switch to linux because it's stable and never does anything to rock the boat. sigh. It's comments like this that convince me that I should never tell anyone about what I'm developing, how it works and what effects it might have. Anything you say will be used against you. I'm excited by these further stability and security enhancing changes. However Genadijus only asked questions. He did not make a statement. Seems like pretty innocent questions to me that are easily answered here by those that know. And what is wrong with that? Shane
Re: 3.8 beta requests
I am not sure if this is related. But when I code assembly to pass a double precision floating point value (%xmm0) to printf, my program will crash without a stack frame. I am fine for passing strings and integers. Here's the simple code: .section .data str: .string %f\n test: .float 2.5 .section .text .extern printf .global main main: push %rbp # set-up stack frame movq %rsp, %rbp# will fault without this movl $str, %edi movl $test, %eax cvtss2sd (%rax), %xmm0 movq $1, %rax call printf movq $1, %rax xorq %rdi, %rdi syscall If I remove the stack frame, this code will fault every time. Now, according to the amd64 ABI, I shouldn't need a stack frame. Now, gcc compiles with stack frames, but this does appear to be a memory bug. I'm just not sure where to go next to research this further. Here's my dmesg: OpenBSD 3.8-beta (GENERIC) #210: Sat Aug 13 20:20:15 MDT 2005 [EMAIL PROTECTED]:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 1073278976 (1048124K) avail mem = 909148160 (887840K) using 22937 buffers containing 107536384 bytes (105016K) of memory mainbus0 (root) cpu0 at mainbus0: (uniprocessor) cpu0: AMD Athlon(tm) 64 Processor 3000+, 1808.55 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 16-way L2 cache cpu0: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative cpu0: DTLB 32 4KB entries fully associative, 8 4MB entries fully associative pci0 at mainbus0 bus 0: configuration mode 1 Nvidia nForce4 DDR rev 0xa3 at pci0 dev 0 function 0 not configured Nvidia nForce4 ISA rev 0xa3 at pci0 dev 1 function 0 not configured Nvidia nForce4 SMBus rev 0xa2 at pci0 dev 1 function 1 not configured ohci0 at pci0 dev 2 function 0 Nvidia nForce4 USB rev 0xa2: irq 10, version 1.0, legacy support usb0 at ohci0: USB revision 1.0 uhub0 at usb0 uhub0: Nvidia OHCI root hub, rev 1.00/1.00, addr 1 uhub0: 10 ports with 10 removable, self powered ehci0 at pci0 dev 2 function 1 Nvidia nForce4 USB rev 0xa3: irq 11 usb1 at ehci0: USB revision 2.0 uhub1 at usb1 uhub1: Nvidia EHCI root hub, rev 2.00/1.00, addr 1 uhub1: 10 ports with 10 removable, self powered auich0 at pci0 dev 4 function 0 Nvidia nForce4 AC97 rev 0xa2: irq 11, nForce4 AC97 ac97: codec id 0x414c4760 (Avance Logic ALC655) audio0 at auich0 pciide0 at pci0 dev 6 function 0 Nvidia nForce4 IDE rev 0xa2: DMA, channel 0 configured to compatibility, channel 1 configured to compatibility pciide0: channel 0 disabled (no drives) atapiscsi0 at pciide0 channel 1 drive 0 scsibus0 at atapiscsi0: 2 targets cd0 at scsibus0 targ 0 lun 0: HL-DT-ST, DVDRAM GSA-4163B, A103 SCSI0 5/cdrom removable cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2 pciide1 at pci0 dev 7 function 0 Nvidia nForce4 SATA 1 rev 0xa3: DMA (unsupported), channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide1: using irq 10 for native-PCI interrupt wd0 at pciide1 channel 0 drive 0: WDC WD360GD-00FLA2 wd0: 16-sector PIO, LBA48, 35304MB, 72303840 sectors pciide1: channel 1 ignored (not responding; disabled or no drives?) pciide2 at pci0 dev 8 function 0 Nvidia nForce4 SATA 2 rev 0xa3: DMA (unsupported), channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide2: using irq 11 for native-PCI interrupt pciide2: channel 0 ignored (not responding; disabled or no drives?) pciide2: channel 1 ignored (not responding; disabled or no drives?) ppb0 at pci0 dev 9 function 0 Nvidia nForce4 PCI-PCI rev 0xa2 pci1 at ppb0 bus 1 vga1 at pci1 dev 5 function 0 ATI Rage XL rev 0x27 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) VIA VT6306 FireWire rev 0x80 at pci1 dev 6 function 0 not configured Nvidia CK804 LAN rev 0xa3 at pci0 dev 10 function 0 not configured ppb1 at pci0 dev 11 function 0 Nvidia nForce4 PCIE rev 0xa3 pci2 at ppb1 bus 2 ppb2 at pci0 dev 12 function 0 Nvidia nForce4 PCIE rev 0xa3 pci3 at ppb2 bus 3 ppb3 at pci0 dev 13 function 0 Nvidia nForce4 PCIE rev 0xa3 pci4 at ppb3 bus 4 bge0 at pci4 dev 0 function 0 Broadcom BCM5721 rev 0x11, BCM5750 B1 (0x4101): irq 5 address 00:e0:81:56:8f:66 brgphy0 at bge0 phy 1: BCM5750 10/100/1000baseT PHY, rev. 0 ppb4 at pci0 dev 14 function 0 Nvidia nForce4 PCIE rev 0xa3 pci5 at ppb4 bus 5 pchb0 at pci0 dev 24 function 0 AMD AMD64 HyperTransport rev 0x00 pchb1 at pci0 dev 24 function 1 AMD AMD64 Address Map rev 0x00 pchb2 at pci0 dev 24 function 2 AMD AMD64 DRAM Cfg rev 0x00 pchb3 at pci0 dev 24 function 3 AMD AMD64 Misc Cfg rev 0x00 isa0 at mainbus0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pmsi0 at pckbc0 (aux slot)
Re: 3.8 beta requests
Your mail has nothing to do with the 3.8 release, nor with testing our code, nor with the malloc stuff I posted. You are hijacking yet another thread with your broken code, and it is quite frankly getting boring. I am not sure if this is related. But when I code assembly to pass a double precision floating point value (%xmm0) to printf, my program will crash without a stack frame. I am fine for passing strings and integers. Here's the simple code: .section .data str: .string %f\n test: .float 2.5 .section .text .extern printf .global main main: push %rbp # set-up stack frame movq %rsp, %rbp# will fault without this movl $str, %edi movl $test, %eax cvtss2sd (%rax), %xmm0 movq $1, %rax call printf movq $1, %rax xorq %rdi, %rdi syscall If I remove the stack frame, this code will fault every time. Now, according to the amd64 ABI, I shouldn't need a stack frame. Now, gcc compiles with stack frames, but this does appear to be a memory bug. I'm just not sure where to go next to research this further. Here's my dmesg: OpenBSD 3.8-beta (GENERIC) #210: Sat Aug 13 20:20:15 MDT 2005 [EMAIL PROTECTED]:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 1073278976 (1048124K) avail mem = 909148160 (887840K) using 22937 buffers containing 107536384 bytes (105016K) of memory mainbus0 (root) cpu0 at mainbus0: (uniprocessor) cpu0: AMD Athlon(tm) 64 Processor 3000+, 1808.55 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 16-way L2 cache cpu0: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative cpu0: DTLB 32 4KB entries fully associative, 8 4MB entries fully associative pci0 at mainbus0 bus 0: configuration mode 1 Nvidia nForce4 DDR rev 0xa3 at pci0 dev 0 function 0 not configured Nvidia nForce4 ISA rev 0xa3 at pci0 dev 1 function 0 not configured Nvidia nForce4 SMBus rev 0xa2 at pci0 dev 1 function 1 not configured ohci0 at pci0 dev 2 function 0 Nvidia nForce4 USB rev 0xa2: irq 10, version 1.0, legacy support usb0 at ohci0: USB revision 1.0 uhub0 at usb0 uhub0: Nvidia OHCI root hub, rev 1.00/1.00, addr 1 uhub0: 10 ports with 10 removable, self powered ehci0 at pci0 dev 2 function 1 Nvidia nForce4 USB rev 0xa3: irq 11 usb1 at ehci0: USB revision 2.0 uhub1 at usb1 uhub1: Nvidia EHCI root hub, rev 2.00/1.00, addr 1 uhub1: 10 ports with 10 removable, self powered auich0 at pci0 dev 4 function 0 Nvidia nForce4 AC97 rev 0xa2: irq 11, nForce4 AC97 ac97: codec id 0x414c4760 (Avance Logic ALC655) audio0 at auich0 pciide0 at pci0 dev 6 function 0 Nvidia nForce4 IDE rev 0xa2: DMA, channel 0 configured to compatibility, channel 1 configured to compatibility pciide0: channel 0 disabled (no drives) atapiscsi0 at pciide0 channel 1 drive 0 scsibus0 at atapiscsi0: 2 targets cd0 at scsibus0 targ 0 lun 0: HL-DT-ST, DVDRAM GSA-4163B, A103 SCSI0 5/cdrom removable cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2 pciide1 at pci0 dev 7 function 0 Nvidia nForce4 SATA 1 rev 0xa3: DMA (unsupported), channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide1: using irq 10 for native-PCI interrupt wd0 at pciide1 channel 0 drive 0: WDC WD360GD-00FLA2 wd0: 16-sector PIO, LBA48, 35304MB, 72303840 sectors pciide1: channel 1 ignored (not responding; disabled or no drives?) pciide2 at pci0 dev 8 function 0 Nvidia nForce4 SATA 2 rev 0xa3: DMA (unsupported), channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide2: using irq 11 for native-PCI interrupt pciide2: channel 0 ignored (not responding; disabled or no drives?) pciide2: channel 1 ignored (not responding; disabled or no drives?) ppb0 at pci0 dev 9 function 0 Nvidia nForce4 PCI-PCI rev 0xa2 pci1 at ppb0 bus 1 vga1 at pci1 dev 5 function 0 ATI Rage XL rev 0x27 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) VIA VT6306 FireWire rev 0x80 at pci1 dev 6 function 0 not configured Nvidia CK804 LAN rev 0xa3 at pci0 dev 10 function 0 not configured ppb1 at pci0 dev 11 function 0 Nvidia nForce4 PCIE rev 0xa3 pci2 at ppb1 bus 2 ppb2 at pci0 dev 12 function 0 Nvidia nForce4 PCIE rev 0xa3 pci3 at ppb2 bus 3 ppb3 at pci0 dev 13 function 0 Nvidia nForce4 PCIE rev 0xa3 pci4 at ppb3 bus 4 bge0 at pci4 dev 0 function 0 Broadcom BCM5721 rev 0x11, BCM5750 B1 (0x4101): irq 5 address 00:e0:81:56:8f:66 brgphy0 at bge0 phy 1: BCM5750 10/100/1000baseT PHY, rev. 0 ppb4 at pci0 dev 14 function 0 Nvidia nForce4 PCIE rev 0xa3 pci5 at ppb4 bus 5 pchb0 at pci0 dev 24 function 0 AMD AMD64 HyperTransport rev 0x00 pchb1 at pci0 dev 24 function 1 AMD AMD64 Address Map rev 0x00 pchb2 at pci0 dev 24 function 2 AMD AMD64 DRAM Cfg rev 0x00 pchb3 at pci0 dev 24
Re: 3.8 beta requests
On Mon, 22 Aug 2005 17:33:40 -0600 Theo de Raadt [EMAIL PROTECTED] wrote: We are heading towards making the real 3.8 release soonish. I was wondering, when can we start pre-ordering our cd-sets? Cheers, Jasper -- Security is decided by quality -- Theo de Raadt
Re: 3.8 beta requests
We are heading towards making the real 3.8 release soonish. I was wondering, when can we start pre-ordering our cd-sets? We normally setup pre-orders 1 month before. We might do it a bit earlier... dunno. But it is hard to do when artwork is not final yet :)
Re: 3.8 beta requests
On Tue, 23 Aug 2005 01:37:12 -0600 Theo de Raadt [EMAIL PROTECTED] wrote: We are heading towards making the real 3.8 release soonish. I was wondering, when can we start pre-ordering our cd-sets? We normally setup pre-orders 1 month before. We might do it a bit earlier... dunno. But it is hard to do when artwork is not final yet :) I wonder what the theme for this release will be... Jasper -- Security is decided by quality -- Theo de Raadt
Re: 3.8 beta requests
2005/8/23, imEnsion [EMAIL PROTECTED]: snip I wonder what the theme for this release will be... /snip hopefully not something political *cough* the 3.4 release https://https.openbsd.org/images/poster10.jpg I really really liked that one.
Re: 3.8 beta requests
On 8/23/05, Theo de Raadt [EMAIL PROTECTED] wrote: This release will bring a lot of new ideas from us. One of them in particular is somewhat risky. First off: I like the idea. The technical merit is obvious. I have a question regarding the timing, though. Is there a particular reason to go ahead with these changes before a freeze? An alternative, going ahead in the first snapshot after branching 3.8, may provide more time to test and get reports on problems. That said, I'll happily test snapshots. Ending up with better software is worth biting a bullet. Cheers, Rogier -- If you don't know where you're going, any road will get you there.
Re: 3.8 beta requests
On Tue, 23 Aug 2005 01:37:12 -0600, Theo de Raadt wrote: We are heading towards making the real 3.8 release soonish. I was wondering, when can we start pre-ordering our cd-sets? We normally setup pre-orders 1 month before. We might do it a bit earlier... dunno. But it is hard to do when artwork is not final yet :) Aw, c'mon, just draw a blank square with Artwork coming soon! in it. I am going to buy my usual 2 copies anyway. I never bought it just because of the graphics, but you guys knew that anyway. At least my copies will beat out the delivery to those who need to see the cover to decide they need 3.8 ;-) From the land down under: Australia. Do we look umop apisdn from up over? Do NOT CC me - I am subscribed to the list. Replies to the sender address will fail except from the list-server.
Re: 3.8 beta requests
On Tue, Aug 23, 2005 at 01:32:11PM +0200, Rogier Krieger wrote: On 8/23/05, Theo de Raadt [EMAIL PROTECTED] wrote: This release will bring a lot of new ideas from us. One of them in particular is somewhat risky. First off: I like the idea. The technical merit is obvious. I have a question regarding the timing, though. Is there a particular reason to go ahead with these changes before a freeze? An alternative, going ahead in the first snapshot after branching 3.8, may provide more time to test and get reports on problems. No disrepect, but you don't know what you're talking about. In particular, you have no idea when this did get activated. If you're routinely running snapshots, you've been running this change for longer than you think.
Re: 3.8 beta requests
...on Tue, Aug 23, 2005 at 09:42:02AM +0200, J. Lievisse Adriaanse wrote: I wonder what the theme for this release will be... Something like we help making your software more secure - by default? (Ok, it's not more secure, but more correct, probably...) Generally I think it's a really good idea to go ahead with this - everything that helps developing software with less errors is a step forward, even if some programmers (and users) may hate it. Alex.
Re: 3.8 beta requests
On 8/23/05, Theo de Raadt [EMAIL PROTECTED] wrote: This release will bring a lot of new ideas from us. One of them in particular is somewhat risky. First off: I like the idea. The technical merit is obvious. I have a question regarding the timing, though. Is there a particular reason to go ahead with these changes before a freeze? An alternative, going ahead in the first snapshot after branching 3.8, may provide more time to test and get reports on problems. Your timeline is entirely wrong. These changes have been worked on for almost 3 years now. And they went in right after the tree unlocked after 3.7. The diffs just had a full test cycle by -current users for a few months already; and lots of bugs got fixed along the way.
Re: 3.8 beta requests
J. Lievisse Adriaanse wrote: On Tue, 23 Aug 2005 01:37:12 -0600 Theo de Raadt [EMAIL PROTECTED] wrote: We are heading towards making the real 3.8 release soonish. I was wondering, when can we start pre-ordering our cd-sets? We normally setup pre-orders 1 month before. We might do it a bit earlier... dunno. But it is hard to do when artwork is not final yet :) I wonder what the theme for this release will be... Maybe a slogan along the lines of, Is your software good enough for OpenBSD!! Perhaps it could be worked into the release's theme. Or a Good Housekeeping Seal of Approval like version of Puffy. I'm not in the publicity game, and for good reason, but it seems a good opportunity for some positive publicity. Some may try and spin the broken applications against OpenBSD. Make sure the OpenBSD story is out there first, loud and clear. Ray
Re: 3.8 beta requests
On 8/23/05, Theo de Raadt [EMAIL PROTECTED] wrote: These changes have been worked on for almost 3 years now. And they went in right after the tree unlocked after 3.7. Thanks for setting me straight. It only means that, at least for my systems, the transition has been pretty painless so far. Cheers, Rogier -- If you don't know where you're going, any road will get you there.
Re: 3.8 beta requests
You've got to use your head, otherwise you'll stick your neck out and
say stupid things.
Of course not. HOW CAN IT? Get real! The hardware is STILL only
providing permissions at the page level!
Apparently the new malloc(3) implementation doesn't stop me from writing past
the end of buffer as long as I am inside the last page.
(Please forgive me beforehand if I am missing something too obvious)
consider the following program:
// We just want to see how far after end of allocated buffer we can go.
// Allocate a buffer, then try to read past it, see how far we can go before
// System notices.
// myhandler should tell us about this. If you don't trust it just disable and
// examine the core file created.
//
#include stdio.h
#include sys/types.h
#include sys/mman.h
#include signal.h
#define PAGE_SIZE 4096
char *s;
int i;
int size;
void myhandler(int sig)
{
printf(Caught Signal %d\n, sig);
printf(s=0x%x, size=%d, i=%d\n, s, size, i);
perror(last err condition);
exit(0);
}
main(int ac, char *av[])
{
if (ac != 2) {
printf(%s size\n, av[0]);
exit(1);
}
size = atoi(av[1]);
if (size 4096) {
printf (size should be larger than 4K.\n);
exit(1);
}
signal(SIGSEGV, myhandler);
s = (char *)malloc(size);
for (i = 0; i 2 * size - 1; i++)
s[i] = (char )i;
free(s);
}
and here is the execution (a very recent install as you can see):
#dmesg | head -6
OpenBSD 3.8-beta (GENERIC.MP) #277: Mon Aug 22 23:04:26 MDT 2005
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel Pentium III (GenuineIntel 686-class) 869 MHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,ME
real mem = 804823040 (785960K)
avail mem = 727044096 (710004K)
# gcc malloc-test.c
# ./a.out 4096
Caught Signal 11
s=0x8a19d000, size=4096, i=4096
last err condition: Undefined error: 0
# ./a.out 4097
Caught Signal 11
s=0x7f0eb000, size=4097, i=8192
last err condition: Undefined error: 0
#
Is this the way it is supposed to be?
cheers,
Masoud Sharbiani
On Mon, Aug 22, 2005 at 05:33:40PM -0600, Theo de Raadt wrote:
We are heading towards making the real 3.8 release soonish. I would
like to ask the community to do lots of testing over the next week if
they can.
This release will bring a lot of new ideas from us. One of them in
particular is somewhat risky. I think it is time to talk about that
one, and let people know what is ahead on our road.
Traditionally, Unix malloc(3) has always just extended the brk,
which means extending the traditional Unix process data segment to
allocate more memory. malloc(3) would simply extend the data segment,
and then calve off little pieces to requesting callers as needed. It
also remembered which pieces were which, so that free(3) could do it's
job.
The way this was always done in Unix has had a number of consequences,
some of which we wanted to get rid of. In particular, malloc free
have not been able to provide strong protection against overflows or
other corruption.
Our malloc implementation is a lot more resistant (than Linux) to
heap overflows in the malloc arena, but we wanted to improve things
even more.
Starting a few months ago, the following changes were made:
- We made the mmap(2) system call return random memory addresses. As well
the kernel ensures that two objects are not mapped next to each other;
in effect, this creates unallocated memory which we call a guard page.
- We have changed malloc(3) to use mmap(2) instead of extending the data
segment via brk()
- We also changed free(3) to return memory to the kernel, un-allocating
them out of the process.
- As before, objects smaller than a page are allocated within shared
pages that malloc(3) maintains. But their allocation is now somewhat
randomized as well.
- A number of other similar changes which are too dangerous for normal
software or cause too much of a slowdown are available as malloc options
as described in the manual page. These are very powerful for debugging
buggy applications.
Other results:
- When you free an object that is = 1 page in size, it is actually
returned to the system. Attempting to read or write to it after
you free is no longer acceptable. That memory is unmapped. You get
a SIGSEGV.
- For a decade and a bit, we have been fixing software for buffer overflows.
Now we are finding a lot of software that reads before the start of the
buffer, or reads too far off the end of the buffer. You get a SIGSEGV.
To some of you, this will sound like what the Electric Fence toolkit
used to be for. But these features are enabled by default. Electric
Fence was also very slow. It took nearly 3 years to write these
OpenBSD changes since
Re: 3.8 beta requests
Hello Theo,
Apparently the new malloc(3) implementation doesn't stop me from writing past
the end of buffer as long as I am inside the last page.
(Please forgive me beforehand if I am missing something too obvious)
consider the following program:
// We just want to see how far after end of allocated buffer we can go.
// Allocate a buffer, then try to read past it, see how far we can go before
// System notices.
// myhandler should tell us about this. If you don't trust it just disable and
// examine the core file created.
//
#include stdio.h
#include sys/types.h
#include sys/mman.h
#include signal.h
#define PAGE_SIZE 4096
char *s;
int i;
int size;
void myhandler(int sig)
{
printf(Caught Signal %d\n, sig);
printf(s=0x%x, size=%d, i=%d\n, s, size, i);
perror(last err condition);
exit(0);
}
main(int ac, char *av[])
{
if (ac != 2) {
printf(%s size\n, av[0]);
exit(1);
}
size = atoi(av[1]);
if (size 4096) {
printf (size should be larger than 4K.\n);
exit(1);
}
signal(SIGSEGV, myhandler);
s = (char *)malloc(size);
for (i = 0; i 2 * size - 1; i++)
s[i] = (char )i;
free(s);
}
and here is the execution (a very recent install as you can see):
#dmesg | head -6
OpenBSD 3.8-beta (GENERIC.MP) #277: Mon Aug 22 23:04:26 MDT 2005
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel Pentium III (GenuineIntel 686-class) 869 MHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,ME
real mem = 804823040 (785960K)
avail mem = 727044096 (710004K)
# gcc malloc-test.c
# ./a.out 4096
Caught Signal 11
s=0x8a19d000, size=4096, i=4096
last err condition: Undefined error: 0
# ./a.out 4097
Caught Signal 11
s=0x7f0eb000, size=4097, i=8192
last err condition: Undefined error: 0
#
Is this the way it is supposed to be?
cheers,
Masoud Sharbiani
On Mon, Aug 22, 2005 at 05:33:40PM -0600, Theo de Raadt wrote:
We are heading towards making the real 3.8 release soonish. I would
like to ask the community to do lots of testing over the next week if
they can.
This release will bring a lot of new ideas from us. One of them in
particular is somewhat risky. I think it is time to talk about that
one, and let people know what is ahead on our road.
Traditionally, Unix malloc(3) has always just extended the brk,
which means extending the traditional Unix process data segment to
allocate more memory. malloc(3) would simply extend the data segment,
and then calve off little pieces to requesting callers as needed. It
also remembered which pieces were which, so that free(3) could do it's
job.
The way this was always done in Unix has had a number of consequences,
some of which we wanted to get rid of. In particular, malloc free
have not been able to provide strong protection against overflows or
other corruption.
Our malloc implementation is a lot more resistant (than Linux) to
heap overflows in the malloc arena, but we wanted to improve things
even more.
Starting a few months ago, the following changes were made:
- We made the mmap(2) system call return random memory addresses. As well
the kernel ensures that two objects are not mapped next to each other;
in effect, this creates unallocated memory which we call a guard page.
- We have changed malloc(3) to use mmap(2) instead of extending the data
segment via brk()
- We also changed free(3) to return memory to the kernel, un-allocating
them out of the process.
- As before, objects smaller than a page are allocated within shared
pages that malloc(3) maintains. But their allocation is now somewhat
randomized as well.
- A number of other similar changes which are too dangerous for normal
software or cause too much of a slowdown are available as malloc options
as described in the manual page. These are very powerful for debugging
buggy applications.
Other results:
- When you free an object that is = 1 page in size, it is actually
returned to the system. Attempting to read or write to it after
you free is no longer acceptable. That memory is unmapped. You get
a SIGSEGV.
- For a decade and a bit, we have been fixing software for buffer overflows.
Now we are finding a lot of software that reads before the start of the
buffer, or reads too far off the end of the buffer. You get a SIGSEGV.
To some of you, this will sound like what the Electric Fence toolkit
used to be for. But these features are enabled by default. Electric
Fence was also very slow. It took nearly 3 years to write these
OpenBSD changes since performance was a serious consideration. (Early
versions caused a nearly 50% slowdown).
Our changes have tremendous benefits, but until some bugs in external
packages are found and fixed, there are some risks as well. Some
software making incorrect assumptions will be running into these new
security technologies.
I
Re: 3.8 beta requests
On 8/24/05, Theo de Raadt [EMAIL PROTECTED] wrote: You've got to use your head, otherwise you'll stick your neck out and say stupid things. Of course not. HOW CAN IT? Get real! The hardware is STILL only providing permissions at the page level! just one quick question. where do I actually learn more about page, buffer, malloc etc?? Is this book enough? http://www.amazon.com/exec/obidos/ISBN%3D0201549794/openbsdA/104-8401808-3342305 or are there other good books out there? Thankyou so much Theo and others for your efforts to advance quality and security :-) And more over Thanks a lot for your patience. Good luck ahead. Take care Kind regards Siju
Re: 3.8 beta requests
On Wed, 24 Aug 2005, Siju George wrote: just one quick question. where do I actually learn more about page, buffer, malloc etc?? Is this book enough? http://www.amazon.com/exec/obidos/ISBN%3D0201549794/openbsdA/104-8401808-3342305 or are there other good books out there? it's useful. the dinosaur book, _operating system concepts_, is also recommended. -- And that's why I stopped reading the big newspapers.
3.8 beta requests
We are heading towards making the real 3.8 release soonish. I would like to ask the community to do lots of testing over the next week if they can. This release will bring a lot of new ideas from us. One of them in particular is somewhat risky. I think it is time to talk about that one, and let people know what is ahead on our road. Traditionally, Unix malloc(3) has always just extended the brk, which means extending the traditional Unix process data segment to allocate more memory. malloc(3) would simply extend the data segment, and then calve off little pieces to requesting callers as needed. It also remembered which pieces were which, so that free(3) could do it's job. The way this was always done in Unix has had a number of consequences, some of which we wanted to get rid of. In particular, malloc free have not been able to provide strong protection against overflows or other corruption. Our malloc implementation is a lot more resistant (than Linux) to heap overflows in the malloc arena, but we wanted to improve things even more. Starting a few months ago, the following changes were made: - We made the mmap(2) system call return random memory addresses. As well the kernel ensures that two objects are not mapped next to each other; in effect, this creates unallocated memory which we call a guard page. - We have changed malloc(3) to use mmap(2) instead of extending the data segment via brk() - We also changed free(3) to return memory to the kernel, un-allocating them out of the process. - As before, objects smaller than a page are allocated within shared pages that malloc(3) maintains. But their allocation is now somewhat randomized as well. - A number of other similar changes which are too dangerous for normal software or cause too much of a slowdown are available as malloc options as described in the manual page. These are very powerful for debugging buggy applications. Other results: - When you free an object that is = 1 page in size, it is actually returned to the system. Attempting to read or write to it after you free is no longer acceptable. That memory is unmapped. You get a SIGSEGV. - For a decade and a bit, we have been fixing software for buffer overflows. Now we are finding a lot of software that reads before the start of the buffer, or reads too far off the end of the buffer. You get a SIGSEGV. To some of you, this will sound like what the Electric Fence toolkit used to be for. But these features are enabled by default. Electric Fence was also very slow. It took nearly 3 years to write these OpenBSD changes since performance was a serious consideration. (Early versions caused a nearly 50% slowdown). Our changes have tremendous benefits, but until some bugs in external packages are found and fixed, there are some risks as well. Some software making incorrect assumptions will be running into these new security technologies. I discussed this in talks I have given before: I said that we were afraid to go ahead with guard pages, because a lot of software is just written to such low standards. Applications over-read memory all the time, go 1 byte too far, read 1 byte too early, access memory after free, etc etc etc. Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. Two examples: Over the last two months, some OpenBSD users noticed that the X server was crashing occasionally. Two bugs have been diagnosed and fixed by us. One was a use-after-free bug in the X shared library linker. The other was a buffer-over-read bug deep down in the very lowest level fb* pixmap compositing routines. The latter bug in particular was very difficult to diagnose and fix, and is about 10 years old. We have found other bugs like this in other external software, and even a few in the base OpenBSD tree (though those were found a while back, even as we started experimenting with the new malloc code). I would bet money that the X fb* bug has crashed Linux (and other) X servers before. It is just that it was very rare, and noone ever chased it. The new malloc we have just makes code get lucky less often, which lets us get to the source of a bug easier. As a programmer, I appreciate anything which makes bugs easier to reproduce. We expect that our malloc will find more bugs in software, and this might hurt our user community in the short term. We know that what this new malloc is doing is perfectly legal, but that realistically some open source software is of such low quality that it is just not ready for these things to happen. We ask our users to help us uncover and fix more of these bugs in applications. Some will even be exploitable. Instead of saying that OpenBSD is busted in this regard, please realize that the software which is crashing is showing how shoddily it was written. Then help us fix it. For everyone.. not just OpenBSD users.
Re: 3.8 beta requests
Am Dienstag, 23. August 2005 01:33 CEST schrieb Theo de Raadt: [*snip lot of interesting stuff beond my scope*] We ask our users to help us uncover and fix more of these bugs in applications. Some will even be exploitable. Instead of saying that OpenBSD is busted in this regard, please realize that the software which is crashing is showing how shoddily it was written. Then help us fix it. For everyone.. not just OpenBSD users. I really like the idea you're describing here. It sounds really beneficial for everyone who doesn't want to play with various implementations to find out the one which works for him (is secure enough), but to have a standardized system wich one can higly rely on; Without patching, recompiling aso. My header blabs my favourite OS, but not for security related systems. And in my opinion you're doing an important step towards best security one can have with still acceptable interoperatibility! I'd guess your users won't be upset because several new(in fact very old) bugs causes crashes, they'll appreciate your foresight. Not to mention the authors of the code;) Best regards -Harry [demime 1.01d removed an attachment of type application/pgp-signature]
Re: 3.8 beta requests
Theo de Raadt wrote: We are heading towards making the real 3.8 release soonish. I would like to ask the community to do lots of testing over the next week if they can. Excellent! Is this is enabled in the current snapshot? Do I need to set any flags in malloc.conf?
Re: 3.8 beta requests
On Monday 22 August 2005 18:33, Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. To paraphrase: I would remind you that extremism in the defense of OpenBSD integrity is no vice! And let me remind you also that moderation in the pursuit of OpenBSD security is no virtue! with a nod to the late Senator Barry Goldwater. Do It! Dave Feustel -- Tired of having to defend against Malware? (You know: trojans, viruses, SPYWARE, worms and popups) Then Switch to OpenBSD with a KDE desktop!!!
Re: 3.8 beta requests
theo, We ask our users to help us uncover and fix more of these bugs in applications. Some will even be exploitable. Instead of saying that OpenBSD is busted in this regard, please realize that the software which is crashing is showing how shoddily it was written. Then help us fix it. For everyone.. not just OpenBSD users. i think these are great ideas, but is there a way to mitigate program breakage if you need to use a given port or program compiled from source? so if something bugs out and you just want it to get lucky for the time being, could you revert to the usual Unix behavior for mmap and such? i think having a flag you could set to disable the new behavior would be a good idea. it may very well be that what i suggest is not doable due to the low-level nature of the functions in question. just a thought. cheers, jake
Re: 3.8 beta requests
We ask our users to help us uncover and fix more of these bugs in applications. Some will even be exploitable. Instead of saying that OpenBSD is busted in this regard, please realize that the software which is crashing is showing how shoddily it was written. Then help us fix it. For everyone.. not just OpenBSD users. i think these are great ideas, but is there a way to mitigate program breakage if you need to use a given port or program compiled from source? No. Find and fix the bug. so if something bugs out and you just want it to get lucky for the time being, could you revert to the usual Unix behavior for mmap and such? No. i think having a flag you could set to disable the new behavior would be a good idea. it may very well be that what i suggest is not doable due to the low-level nature of the functions in question. just a thought. It might be a good idea, but it is just not possible. There are too many pieces.
Re: 3.8 beta requests
Am Dienstag, 23. August 2005 03:49 CEST schrieb Dave Feustel: On Monday 22 August 2005 18:33, Theo de Raadt wrote: Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them. To paraphrase: I would remind you that extremism in the defense of OpenBSD integrity is no vice! And let me remind you also that moderation in the pursuit of OpenBSD security is no virtue! Hmm, that's the way I'd love to be able to say things, but even in my native language I haven't populated such nice statements :( Well spoken, darling ;) -Harry with a nod to the late Senator Barry Goldwater. Do It! Dave Feustel [demime 1.01d removed an attachment of type application/pgp-signature]
Re: 3.8 beta requests
i think these are great ideas, but is there a way to mitigate program breakage if you need to use a given port or program compiled from source? so if something bugs out and you just want it to get lucky for the time being, could you revert to the usual Unix behavior for mmap and such? Fix it!! This benefits all open source code for all OS' i think having a flag you could set to disable the new behavior would be a good idea. it may very well be that what i suggest is not doable due to the low-level nature of the functions in question. just a thought. knobs suck.
Re: 3.8 beta requests
On Aug 22, 2005, at 10:32 PM, Theo de Raadt wrote: i think having a flag you could set to disable the new behavior would be a good idea. it may very well be that what i suggest is not doable due to the low-level nature of the functions in question. just a thought. It might be a good idea, but it is just not possible. There are too many pieces. Not only is it a bad idea, it undermines the goals of the change. This is a good example of why SELinux hasn't been readily accepted (beyond being a suckass piece of bolt-on garbage); it's too easy to just disable it, rather than a) fixing the underlying bad code, or b) learning how to properly use the tool. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net
Re: 3.8 beta requests
-Original Message- From: [EMAIL PROTECTED] on behalf of Theo de Raadt Sent: Mon 8/22/2005 7:33 PM To: [EMAIL PROTECTED] Subject: 3.8 beta requests We are heading towards making the real 3.8 release soonish. I would like to ask the community to do lots of testing over the next week if they can. What is the best way to test? Should we be downloading snapshots daily?
Re: 3.8 beta requests
We are heading towards making the real 3.8 release soonish. I would like to ask the community to do lots of testing over the next week if they can. What is the best way to test? Should we be downloading snapshots daily? Install snapshots. Install snapshot packages. Try using it as if it is the real 3.8. Tell us if things fail.
Re: 3.8 beta requests
On 8/22/05, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: i think having a flag you could set to disable the new behavior would be a good idea. it may very well be that what i suggest is not doable due to the low-level nature of the functions in question. just a thought. To complement the previous arguments against adding more knobs, toggles, switches and crap: Security only works if the secure way is also the easy way. Notice that every production security technology openbsd includes a) is easy to use and b) is on by default. OpenSSH, privsep, strong random numbers, strong crypto... you have to try weaken your system. As others have said, other security technologies have failed/are failing/ are not properly used because they're a) hard to use properly and b) easy to not use. I'll admit it: I'm lazy. Once upon a time a port that I used was broken by an openbsd security technology. I was too lazy to track down the breakage, so I uninstalled the port and went with pretty similar functionality in the base system. Once upon a time there was a program that I used that was unreliable. It crashed, hung, busy waited, etc. for no good reason. I crashed it on openbsd a few times, found the bugs, and reported them (with patches) to the author. Now it's stable. As Theo and others will no doubt tell you the right thing to do is fix the buggy program, rather than running a provably buggy program with significant memory management issues. Now did you notice how Theo said that the new mmap, guard pages, and other things stomp on errors as small as a single byte. Remember that ssh bug? That was a one byte overflow. Explain to us why turning off the security systems to keep a buggy program from being terminated is a good thing? CK -- GDB has a 'break' feature; why doesn't it have 'fix' too?
