Re: malloc freelists
Okay, the question is: why 4 ? why not 3 ? or 2 ? or 8 ? Where do you stop ? how did you figure out that 4 was better ? This looks a bit like hey, let's make our own crypto code, it ought to work just fine, right ?
Re: malloc freelists
Sorry, badly phrased reply. I didn't mean to imply it was a bad idea, but you didn't explain at all why 4, and not 3 or 6, or 42 ? If it's good with 4, it ought to be better with more, right ? any data point or rationale for choosing 4 ?
Re: libc: #define to remove support for %n from printf(3)?
On Sat, May 03, 2014 at 11:28:16AM +0200, Reyk Floeter wrote: On Fri, May 02, 2014 at 06:50:04PM -0600, Bob Beck wrote: What's their hangup with %n? We normally don't like polluting the world with #ifdef OPENSSL_NO_PERCENT_N... We normally nuke stuff like that Well, it is an evil thing that is rarely used and well-known for some format string vulnerabilities. So either adding this #define or removing it from our libc wouldn't be a bad idea. But it is in POSIX, right? At a first look, I only found it in a few places of our tree where it could be removed: gcc, texinfo, curses, libform, sqlite, less, tmux, unbound, nsd Not couting ports, but some software already seems to check if %n is available to do a fallback otherwise. Reyk And you do expect the fallback to be of good quality ? I think that the same rationale that lead to including some crypto code in openssl applies there: if we don't provide it, people will roll their own posixy-printf, and we know how well that turns out in general... (including stuff that peeks and duplicates the internals of FILE, like I'll be happy if I *never* get to fix this kind of code again).
Re: libc: #define to remove support for %n from printf(3)?
On Sat, May 03, 2014 at 01:00:28PM +0200, Henning Brauer wrote: * Marc Espie es...@nerim.net [2014-05-03 11:50]: On Sat, May 03, 2014 at 11:28:16AM +0200, Reyk Floeter wrote: On Fri, May 02, 2014 at 06:50:04PM -0600, Bob Beck wrote: What's their hangup with %n? We normally don't like polluting the world with #ifdef OPENSSL_NO_PERCENT_N... We normally nuke stuff like that Well, it is an evil thing that is rarely used and well-known for some format string vulnerabilities. So either adding this #define or removing it from our libc wouldn't be a bad idea. But it is in POSIX, right? At a first look, I only found it in a few places of our tree where it could be removed: gcc, texinfo, curses, libform, sqlite, less, tmux, unbound, nsd Not couting ports, but some software already seems to check if %n is available to do a fallback otherwise. And you do expect the fallback to be of good quality ? I think that the same rationale that lead to including some crypto code in openssl applies there: if we don't provide it, people will roll their own posixy-printf, and we know how well that turns out in general... (including stuff that peeks and duplicates the internals of FILE, like I'll be happy if I *never* get to fix this kind of code again). both of you are kinda missing the point. there is no argument here to remove %n support from our libc, the point is wether we can add a #define to allow people compiling themselves (probably not as part of OpenBSD) to remove it without having to change the code. And since that's not intrusive and doesn't create a portability mess like the one we're dealing with in libssl right now, I don't see a problem with that. No, I was just responding to Reyk, who apparently wanted to take it further. As long as the #ifdef doesn't turn into yet another quagmire, it's obviously okay to have it (coming after the commit, of course).
Re: malloc in libssl/src/apps
On Sun, May 04, 2014 at 12:21:04AM -0700, patrick keshishian wrote: why not use calloc(2)? Because it doesn't exist ? (hint: the 2 in calloc(2) is the manual section. There is no calloc system call, therefore you can't be right. See guenther(2) for a more serious answer).
Re: Support for LC_TIME
On Thu, May 08, 2014 at 12:07:30PM +0200, Stefan Sperling wrote: On Wed, May 07, 2014 at 07:44:51PM +0200, Ingo Schwarze wrote: While LC_CTYPE and LC_COLLATE make some sense, LC_MONETARY, LC_NUMERIC, and LC_TIME are badly overengineered, pointless bloat, causing nothing but surprising, erratic behaviour and portability problems when trying to parse output from programs. I think this should be rejected outright and you should stop wasting your time on it. They make sense for systems that try to provide full i18n. Of course, we don't try to provide i18n, at least not for the base system which is English only. So they don't really make sense *for OpenBSD*. ??? Basic support for that stuff makes sense, as part of a *full* libc. Not surprisingly, Antoine is for providing LC_* support. So am I. This has little to do with base OpenBSD, everything to do with enough stuff to be able to compile reasonable portable software on OpenBSD without needing to patch left and right. As for portability issues: programs stay with the C locale *in any case* unless they do setlocale() right at the start, in which case they explicitly say yes, I want to be localized. So, from that point of view the portability issues are minimal (yes, I'm aware of the can of worms that threads+locale may open). That said, I don't have a general problem with adding other locale categories. I believe LC_TIME would provide a useful testbed for eventually switching all our locales to the localedef format (including LC_CTYPE). Alas, the proposed diff does something else, and unfortunately I don't have enough time for a detailed rabbit hole discussion and review with a lot of back-and-forth that we had when discussing similar diffs in the past. THAT on the other hand is the issue at hand... chronic time shortage to be certain that what we do for locales isn't dangerous...
pkg_add: extract out-of-order archives
I already sent a preliminary version of this patch to selected people. Now that packages use long names when needed, we can pinpoint every file in the archive based on its archived name. This patch changes pkg_add to be able to extract archives that contain elements out-of-order, e.g., match package contents to plist in whatever order. Once the temporary elements are extracted, the actual extract routine will put everything in place, run scripts, etc. This is really important. Test/read/report failures to me. Once this is in, there is another patch that will allow pkg_create to create out-of-order archives. The neat effect will be to speed up updates, by relegating unchanged files to the end of the archive, so they don't need to be fetched at all. the effect should be drastic on monsters like texlive, where most of the files do not change at all from version to version. Index: OpenBSD/Add.pm === RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/Add.pm,v retrieving revision 1.153 diff -u -p -r1.153 Add.pm --- OpenBSD/Add.pm 20 May 2014 05:55:43 - 1.153 +++ OpenBSD/Add.pm 12 Jun 2014 09:50:44 - @@ -113,7 +113,33 @@ sub perform_extraction $state-{partial} = $handle-{partial}; $state-{archive} = $handle-{location}; $state-{check_digest} = $handle-{plist}{check_digest}; - $state-progress-visit_with_size($handle-{plist}, 'extract', $state); + my ($wanted, $tied) = {}; + $handle-{plist}-find_extractible($state, $wanted, $tied); + my $p = $state-progress-new_sizer($handle-{plist}, $state); + while (my $file = $state-{archive}-next) { + my $e = $tied-{$file-name}; + if (defined $e) { + delete $tied-{$file-name}; + $p-advance($e); + $state-{archive}-skip; + # skip to next; + } + $e = $wanted-{$file-name}; + if (!defined $e) { + $state-Fatal(archive member not found #1, + $file-name); + } + delete $wanted-{$file-name}; + $e-prepare_to_extract($state, $file); + $e-extract($state, $file); + $p-advance($e); + if (keys %$wanted == 0) { + return; + } + } + if (keys %$wanted 0) { + $state-fatal(Truncated archive); + } } my $user_tagged = {}; @@ -181,6 +207,10 @@ sub prepare_for_addition { } +sub find_extractible +{ +} + sub extract { my ($self, $state) = @_; @@ -236,6 +266,16 @@ sub set_modes } } +package OpenBSD::PackingElement::Meta; + +# XXX stuff that's invisible to find_extractible should be considered extracted +# for the most part, otherwise we create broken partial packages +sub find_extractible +{ + my ($self, $state, $wanted, $tied) = @_; + $state-{partial}{$self} = 1; +} + package OpenBSD::PackingElement::ExtraInfo; use OpenBSD::Error; @@ -360,6 +400,18 @@ use File::Basename; use File::Path; use OpenBSD::Temp; +sub find_extractible +{ + my ($self, $state, $wanted, $tied) = @_; + $wanted-{$self-name} = $self; + return; + if ($self-{tied}) { + $tied-{$self-name} = $self; + } else { + $wanted-{$self-name} = $self; + } +} + sub prepare_for_addition { my ($self, $state, $pkgname) = @_; @@ -384,19 +436,11 @@ sub prepare_for_addition sub prepare_to_extract { - my ($self, $state) = @_; + my ($self, $state, $file) = @_; my $fullname = $self-fullname; my $destdir = $state-{destdir}; - my $file=$state-{archive}-next; - if (!defined $file) { - $state-fatal(truncated archive); - } $file-{cwd} = $self-cwd; - if (!$file-check_name($self)) { - $state-fatal(archive does not match #1 != #2, - $file-name, $self-name); - } if (defined $self-{symlink} || $file-isSymLink) { unless (defined $self-{symlink} $file-isSymLink) { $state-fatal(bogus symlink #1, $self-name); @@ -427,14 +471,11 @@ sub prepare_to_extract if (defined $self-{symlink} $state-{do_faked}) { $file-{linkname} = $destdir.$file-{linkname}; } - return $file; } sub extract { - my ($self, $state) = @_; - - my $file = $self-prepare_to_extract($state); + my ($self, $state, $file) = @_; if (defined $self-{link} || defined $self-{symlink}) { $state-{archive}-skip; @@ -483,6 +524,7 @@ sub extract $state-say(extracting #1, $tempname) if $state-verbose = 3; + if (!$file-isFile) { $state-fatal(can't extract #1, it's not a file,
newer patch Re: pkg_add: extract out-of-order archives
Tweaked version. The previous version was working just fine, but this one also marks hardlinks and symlinks as skippable, as should be. It also adjusts the pace of progress slightly (making the meter jump AFTER skipping the entry), and also it does prepare_to_extract systematically (which will recheck links vs their packing-list entry for consistency) I haven't seen any negative report from the previous version. If this one does work too, I intend to commit it sometimes during the week. Index: OpenBSD/Add.pm === RCS file: /build/data/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/Add.pm,v retrieving revision 1.153 diff -u -p -r1.153 Add.pm --- OpenBSD/Add.pm 20 May 2014 05:55:43 - 1.153 +++ OpenBSD/Add.pm 13 Jun 2014 17:14:05 - @@ -113,7 +113,34 @@ sub perform_extraction $state-{partial} = $handle-{partial}; $state-{archive} = $handle-{location}; $state-{check_digest} = $handle-{plist}{check_digest}; - $state-progress-visit_with_size($handle-{plist}, 'extract', $state); + my ($wanted, $tied) = {}; + $handle-{plist}-find_extractible($state, $wanted, $tied); + my $p = $state-progress-new_sizer($handle-{plist}, $state); + while (my $file = $state-{archive}-next) { + my $e = $tied-{$file-name}; + if (defined $e) { + delete $tied-{$file-name}; + $e-prepare_to_extract($state, $file); + $state-{archive}-skip; + $p-advance($e); + # skip to next; + } + $e = $wanted-{$file-name}; + if (!defined $e) { + $state-Fatal(archive member not found #1, + $file-name); + } + delete $wanted-{$file-name}; + $e-prepare_to_extract($state, $file); + $e-extract($state, $file); + $p-advance($e); + if (keys %$wanted == 0) { + return; + } + } + if (keys %$wanted 0) { + $state-fatal(Truncated archive); + } } my $user_tagged = {}; @@ -181,6 +208,10 @@ sub prepare_for_addition { } +sub find_extractible +{ +} + sub extract { my ($self, $state) = @_; @@ -236,6 +267,16 @@ sub set_modes } } +package OpenBSD::PackingElement::Meta; + +# XXX stuff that's invisible to find_extractible should be considered extracted +# for the most part, otherwise we create broken partial packages +sub find_extractible +{ + my ($self, $state, $wanted, $tied) = @_; + $state-{partial}{$self} = 1; +} + package OpenBSD::PackingElement::ExtraInfo; use OpenBSD::Error; @@ -360,6 +401,18 @@ use File::Basename; use File::Path; use OpenBSD::Temp; +sub find_extractible +{ + my ($self, $state, $wanted, $tied) = @_; + $wanted-{$self-name} = $self; + return; + if ($self-{tied} || $self-{link} || $self-{symlink}) { + $tied-{$self-name} = $self; + } else { + $wanted-{$self-name} = $self; + } +} + sub prepare_for_addition { my ($self, $state, $pkgname) = @_; @@ -384,19 +437,11 @@ sub prepare_for_addition sub prepare_to_extract { - my ($self, $state) = @_; + my ($self, $state, $file) = @_; my $fullname = $self-fullname; my $destdir = $state-{destdir}; - my $file=$state-{archive}-next; - if (!defined $file) { - $state-fatal(truncated archive); - } $file-{cwd} = $self-cwd; - if (!$file-check_name($self)) { - $state-fatal(archive does not match #1 != #2, - $file-name, $self-name); - } if (defined $self-{symlink} || $file-isSymLink) { unless (defined $self-{symlink} $file-isSymLink) { $state-fatal(bogus symlink #1, $self-name); @@ -427,14 +472,11 @@ sub prepare_to_extract if (defined $self-{symlink} $state-{do_faked}) { $file-{linkname} = $destdir.$file-{linkname}; } - return $file; } sub extract { - my ($self, $state) = @_; - - my $file = $self-prepare_to_extract($state); + my ($self, $state, $file) = @_; if (defined $self-{link} || defined $self-{symlink}) { $state-{archive}-skip; @@ -483,6 +525,7 @@ sub extract $state-say(extracting #1, $tempname) if $state-verbose = 3; + if (!$file-isFile) { $state-fatal(can't extract #1, it's not a file, $self-stringize); @@ -558,6 +601,10 @@ sub prepare_for_addition if ($s-avail 0) { $s-report_overflow($state, $fname); } +} + +sub find_extractible +{ } sub extract Index: OpenBSD/ProgressMeter.pm
new package format
I've just switched on out-of-order packages, after much testing. What this means: new packages won't be compatible with older pkg_add. Most specifically, the plist order may no longer match the packing-list. - if you see strange pkg_add errors, and your base system is not uptodate, that's your fault. New bulks will exploit that feature: build machines log an history of files in a package. With out-of-order archives, they can take advantage of that to produce packages where the most recently changed files are at the front. For some packages, this can be a drastic performance improvement: some measurements show that as much as half of some packages do not change over a 2 months period. Note that this will speed up dependency, maintainer and other meta info updates even more, as the package extraction may stop right after the meta information...
Re: using -Werror-implicit-function-declaration
On Fri, Jul 11, 2014 at 10:27:57AM -0400, Ted Unangst wrote: On Fri, Jul 11, 2014 at 14:09, Florian Obser wrote: usr.sbin, make sure subdirs of subdirs see usr.sbin/Makefile.inc: Does the same thing as lpr/pac and pppd/pppstats. I have no idea if this is the right way, but it seems to work. Cluebats welcome. I think maybe it would be nicer to create a Makefile.inc in .. and have that include ../../Makefile.inc in some cases? That would work more automagically? Avoiding multiple inclusions is somewhat hard...
Re: PATCH: misc mkstemp and fdopen fixes
On Fri, Jul 11, 2014 at 04:55:36PM +, Doug Hogan wrote: Index: usr.bin/m4/eval.c === RCS file: /cvs/src/usr.bin/m4/eval.c,v retrieving revision 1.72 diff -u -p -d -r1.72 eval.c --- usr.bin/m4/eval.c 28 Apr 2014 12:34:11 - 1.72 +++ usr.bin/m4/eval.c 11 Jul 2014 16:20:25 - @@ -818,8 +818,12 @@ dodiv(int n) char fname[] = _PATH_DIVNAME; if ((fd = mkstemp(fname)) 0 || - (outfile[n] = fdopen(fd, w+)) == NULL) - err(1, %s: cannot divert, fname); + (outfile[n] = fdopen(fd, w+)) == NULL) { + int saved_errno = errno; + if (fd != -1) + unlink(fname); + errc(1, saved_errno, %s: cannot divert, fname); + } if (unlink(fname) == -1) err(1, %s: cannot unlink, fname); } I don't like that part. The logic is a bit wrong. Especially since unlink(fname) is always called for fd != -1, so I feel there should be one single call.
Re: First release of LibreSSL portable is available.
On Fri, Jul 11, 2014 at 11:07:10PM +, Miod Vallat wrote: it. As expected, OPENSSL does the opposite and makes life harder for everyone. Hasn't this been the OpenSSL roadmap since the very beginning? Jury is still out as whether they did it on purpose, or whether it was just a side-effect of the lack of funding, or whether they did it on purpose to later complain about the lack of funding.
HEADS-UP: faster mirrors synching coming your way soon
I've just committed changes to pkg_create that will help mirrors synch by using much less bandwidth. I just ran a final test. Rsynching a full amd64 snapshot now says something like: sent 7,315,796,510 bytes received 40,292,721 bytes 4,517,095.01 bytes/sec total size is 28,752,806,019 speedup is 3.91 A few months ago, after the reorder files in packages, Stuart Henderson commented that this would not help mirrors, but just the end user, which got me thinking... (Reminder: archives are compressed files. rsync does not peek inside the compressed data, so its comparison algorithms don't work so well with them, as the first different byte will change everything for the rest of the archive, so no speed-up for compressed files). I looked at the --rsyncable patch for zlib/gzip, and talked it over with sthen@ and millert@, but pretty soon we discarded that idea. That patch is brittle (every zlib version has got its own flavor of it, with wild differences) and a nightmare to maintain. Plus it won't work at all with other compression formats. The solution was low-tech: simply cut the archive into more gzip chunks (signatures already split the package into two parts, so we know the tools work). I chose 16 files as a simple guideline to experiment with. There were still some discrepancies, such as tar timestamps metadata, which is why those migrated to the plist a few weeks ago (side-effect: the tarball effectively says everything dates back to the epoch... not so bad). I was pleasantly surprised: the size increase is minimal (very much under 1%). I also wacked on gzip timestamps, which don't serve any useful purpose either, especially since the plist signature also contains a timestamp (and that one is signed, so it's ways more trustworthy). Obviously, the first snapshot out will still copy everything. But from the second one, mirror owners should see a difference. To benefit: - mirror owners must now use rsync algorithms. Turn off -W / --whole-file if you were using it. - turn on -y / --fuzzy, as this will track minor package version changes. Note that this only applies to the package snapshots part of OpenBSD. My test was a bit extreme: I did build two snaps with the exact same ports tree, so the similarities are maximal. Nevertheless, there are lots of *huge* packages in the ports tree. So I expect the bandwidth gain to be very significant anyway, especially for fast architectures which turn up one snapshot a week or more. e.g., bandwidth use should be more than halved, I expect.
Re: HEADS-UP: faster mirrors synching coming your way soon
On Mon, Sep 22, 2014 at 09:42:58AM +0200, Marc Espie wrote: I've just committed changes to pkg_create that will help mirrors synch by using much less bandwidth. I just ran a final test. Rsynching a full amd64 snapshot now says something like: sent 7,315,796,510 bytes received 40,292,721 bytes 4,517,095.01 bytes/sec total size is 28,752,806,019 speedup is 3.91 My test was a bit extreme: I did build two snaps with the exact same ports tree, so the similarities are maximal. Nevertheless, there are lots of *huge* packages in the ports tree. So I expect the bandwidth gain to be very significant anyway, especially for fast architectures which turn up one snapshot a week or more. e.g., bandwidth use should be more than halved, I expect. I got a newer snaps, so an actual test on real data. Snaps apart by about 1 week. sent 7,502,610,665 bytes received 40,313,605 bytes 4,561,792.72 bytes/sec total size is 28,788,092,671 speedup is 3.82 Still very good.
Re: Conditional include in make(1)
On Sun, Oct 19, 2014 at 09:25:51AM +0200, Matthieu Herrb wrote: On Thu, Oct 16, 2014 at 08:14:16PM +0200, Mark Kettenis wrote: Hi Marc, Is there a reason why conditional includes (sinclude/-include) aren't enabled in OpenBSD? I'm asking because the Xorg people now use it in one of the xserver Makefile. We could of course try to convince them to revert the change they made. But it is a somewhat useful feature. So, now that espie@ has committed the bit, ok to remove the local change that made the inclusion unconditionnal (and slightly broke make clean in xserver) ? Index: Makefile.am === RCS file: /cvs/OpenBSD/xenocara/xserver/hw/xfree86/Makefile.am,v retrieving revision 1.9 diff -u -r1.9 Makefile.am --- Makefile.am 27 Sep 2014 17:53:01 - 1.9 +++ Makefile.am 19 Oct 2014 07:23:22 - @@ -138,7 +138,7 @@ $(AM_V_GEN)CPP='$(CPP)' AWK='$(AWK)' $(SHELL) $(srcdir)/sdksyms.sh $(top_srcdir) $(CFLAGS) $(AM_CFLAGS) $(AM_CPPFLAGS) SDKSYMS_DEP = sdksyms.dep -include $(SDKSYMS_DEP) +-include $(SDKSYMS_DEP) i2c/libi2c.la: $(AM_V_at)cd i2c $(MAKE) libi2c.la Index: Makefile.in === RCS file: /cvs/OpenBSD/xenocara/xserver/hw/xfree86/Makefile.in,v retrieving revision 1.29 diff -u -r1.29 Makefile.in --- Makefile.in 27 Sep 2014 17:53:01 - 1.29 +++ Makefile.in 19 Oct 2014 07:23:22 - @@ -1116,7 +1116,7 @@ sdksyms.dep sdksyms.c: sdksyms.sh $(AM_V_GEN)CPP='$(CPP)' AWK='$(AWK)' $(SHELL) $(srcdir)/sdksyms.sh $(top_srcdir) $(CFLAGS) $(AM_CFLAGS) $(AM_CPPFLAGS) -include $(SDKSYMS_DEP) +-include $(SDKSYMS_DEP) i2c/libi2c.la: $(AM_V_at)cd i2c $(MAKE) libi2c.la -- Wait for a bit until new snaps have the new make, then sure.
Re: help X11 performance: make sigprocmask(2) SY_NOLOCK
On Wed, Jun 19, 2013 at 01:40:19PM +0200, Martin Pelikan wrote: If you're right that atomic_{clear,set}bits_int is correct and sufficient and actually faster, then all dynamic executables would benefit from this speedup (sigprocmask is used in ld.so(1)). Since on i386 GENERIC these atomic_* things don't emit the LOCK prefix, performance shouldn't be an issue; I'm actually more worried about this bit: - p-p_sigmask = mask ~ sigcantmask; + p-p_sigmask = mask; On the right architecture where a word store isn't atomic enough and with the right compiler that decides to put p_sigmask on an address ending with 0xFFF with 4k-sized pages, we have two problems already. If I read the code correctly, just adding an alignment constraint to pool_init for the proc_pool would fix that 2nd part.
Re: binary integer constants in gcc
On Fri, Jun 21, 2013 at 11:03:16AM +0200, Mark Kettenis wrote: Date: Fri, 21 Jun 2013 10:50:42 +0200 From: Landry Breuil lan...@rhaalovely.net On Fri, Jun 21, 2013 at 10:20:01AM +0200, Mark Kettenis wrote: Well, lots of ports stuff is compiled with newer gcc versions anyway. Actually, not so many: $echo select count(*) from modules where value='gcc4'; | sqlite3/usr/local/share/sqlports 34 And if you rip out all the subpackages, the actual list is: audio/mscore editors/libreoffice lang/classpath lang/luajit net/rtorrent print/cups-filters textproc/pdftk www/mozilla-firefox www/seamonkey www/squid Don't libreoffice and mozilla-firefox account for about half the lines of code that's in ports? ;) Seriously though; I was under the impression that it was a lot more. Thanks for enlightening me Landry. Some of the big ones have moved to llvm (chromium). In actuality, as you know, we're playing linking games, since we mix both libstdc++ from base and libstdc++ from the ports gcc4. The only clean option would be to move all the ports that want C++ to using a single compiler... or to a single libstdc++/libcxxrt...
Re: [Fwd: HTTP2 script problem]
On Sat, Jun 22, 2013 at 12:06:51PM -0700, Ryan Freeman wrote: On Sat, Jun 22, 2013 at 07:00:09PM +0200, Max Power wrote: Hi guys! OpenBSD 5.3/amd64: pkg_add apache-httpd [ok.] next step /etc/rc.d/httpd2 start returns: httpd2(failed) Instead /usr/local/sbin/apachectl2 start It works and load Apache2. Why? Because you keep wasting time emailing a technical list for improvements and patches, VERY SPECIFICALLY NOTED on the page where this address resides. If you spent any time at all reading manual pages, you would have all the questions you have asked answered. STOP POSTING THESE QUESTIONS TO THIS LIST. m...@openbsd.org is the ONLY list you should be posting to, based on all the questions I have seen from you so far. -ryan I will also say that, eventually, we will ask the list caretaker to BAN YOU FROM TECH@. GO TO MISC@ You've been warned three times. NOOB QUESTIONS *DO NOT BELONG* ON TECH@.
Re: Update elf(5)
On Mon, Jul 08, 2013 at 11:36:14PM -0700, Matthew Dempsky wrote: Might be nice to mention that 5.4 'was' the first release that used ELF on all platforms? Starting .Ox 5.4 , all supported platforms use it as the native binary file format. On Mon, Jul 8, 2013 at 11:16 PM, Simon Kuhnle si...@blarzwurst.de wrote: With VAX being an ELF platform, this is no longer true, is it? Regards Simon Index: elf.5 === RCS file: /cvs/src/share/man/man5/elf.5,v retrieving revision 1.20 diff -u -r1.20 elf.5 --- elf.5 17 Jan 2013 21:54:18 - 1.20 +++ elf.5 9 Jul 2013 06:14:54 - @@ -1355,9 +1355,7 @@ .Sh HISTORY .Ox ELF support first appeared in -.Ox 1.2 , -although not all supported platforms use it as the native -binary file format. +.Ox 1.2 . ELF in itself first appeared in .At V . The ELF format is an adopted standard.
Re: Update elf(5)
On Tue, Jul 09, 2013 at 06:09:02PM +0200, Simon Kuhnle wrote: On Tue, Jul 09, 2013 at 12:29:10AM -0700, Matthew Dempsky wrote: On Tue, Jul 9, 2013 at 12:22 AM, Marc Espie es...@nerim.net wrote: Starting .Ox 5.4 , all supported platforms use it as the native binary file format. I think Starting *with* .Ox 5.4, ... but otherwise ok matthew. Updated with your suggestions. Regards Simon Index: elf.5 === RCS file: /cvs/src/share/man/man5/elf.5,v retrieving revision 1.20 diff -u -r1.20 elf.5 --- elf.5 17 Jan 2013 21:54:18 - 1.20 +++ elf.5 9 Jul 2013 16:07:27 - @@ -1355,9 +1355,10 @@ .Sh HISTORY .Ox ELF support first appeared in -.Ox 1.2 , -although not all supported platforms use it as the native -binary file format. +.Ox 1.2 . +Starting with +.Ox 5.4 , +all supported platforms use it as the native binary file format. ELF in itself first appeared in .At V . The ELF format is an adopted standard. Okay
Re: arp rarpd ndp warnings
On Sat, Jul 20, 2013 at 12:40:47AM +0200, Alexander Bluhm wrote: Hi, Enable gcc warnings for arp, rarpd, ndp and fix findings. ok? bluhm Index: usr.sbin/arp/Makefile === RCS file: /data/mirror/openbsd/cvs/src/usr.sbin/arp/Makefile,v retrieving revision 1.3 diff -u -p -u -p -r1.3 Makefile --- usr.sbin/arp/Makefile 29 Mar 2005 23:56:30 - 1.3 +++ usr.sbin/arp/Makefile 19 Jul 2013 22:35:30 - @@ -2,6 +2,7 @@ PROG=arp MAN= arp.4 arp.8 -CFLAGS+= -Wall + +WARNINGS= Yes I may be wrong, but I think the consensus was that WARNINGS was something you turned on when you wanted to check things, but that was too expensive in terms of compiler speed, especially on slow slow architectures ?
Re: Inline assembly is hard...
On Tue, Aug 06, 2013 at 12:37:49AM +0200, Mark Kettenis wrote: The atomic_setbits_int() and atomic_clearbits_int() implementations use stwcx., which touches the condition code register. This means we need to tell GCC that it gets clobbered. Fixes issues with uvm_page_physload() which currently gets miscompiled. ok? Index: atomic.h === RCS file: /cvs/src/sys/arch/powerpc/include/atomic.h,v retrieving revision 1.4 diff -u -p -r1.4 atomic.h --- atomic.h 23 Mar 2011 16:54:36 - 1.4 +++ atomic.h 5 Aug 2013 22:34:00 - @@ -17,7 +17,7 @@ atomic_setbits_int(__volatile unsigned i or %0, %1, %0 \n stwcx. %0, 0, %2 \n bne-1b \n -sync : =r (tmp) : r (v), r (uip) : memory); +sync : =r (tmp) : r (v), r (uip) : cc, memory); } static __inline void @@ -30,7 +30,7 @@ atomic_clearbits_int(__volatile unsigned andc%0, %0, %1 \n stwcx. %0, 0, %2 \n bne-1b \n -sync : =r (tmp) : r (v), r (uip) : memory); +sync : =r (tmp) : r (v), r (uip) : cc, memory); } #endif /* defined(_KERNEL) */ Obviously okay.
Re: Iso image integrity verification
On Wed, Sep 11, 2013 at 05:36:45PM +0300, Valentin Zagura wrote: Thanks for the suggestion, we will probably order the CD. But on the other hand, I hope that you realize that people in some countries (Iran, China, Egypt, Syria) would not have this possibility and they could be more affected by a compromise than we would be (they might probably pay with their lives) and I hope you guys are also thinking of them. Thanks, Valentin Zagura Do your homework. There are specifically companies that deal with OpenBSD in such countries, most specially the ones who can't deal with the US because of embargoes...
Re: more /dev/ugen*
In general, when we don't have enough of a device, we end up with clonable shit or something don't we ?
Re: more /dev/ugen*
On Fri, Sep 13, 2013 at 07:24:27PM +0200, Mark Kettenis wrote: Date: Fri, 13 Sep 2013 17:55:17 +0200 From: Marc Espie es...@nerim.net In general, when we don't have enough of a device, we end up with clonable shit or something don't we ? Not really. Cloneable devices are used to create per-open context. well, it seems to me that having a limit on the number of devices like usb stuff or something is somewhat wasteful. It would probably be nicer if it could adjust automatically (note that I have no idea how much more work this involves besides allocating the structures in the right location... but since usb hot-attaches after boot, I figure most of the work is already there).
Re: /etc/rc.d/rc.subr; prefix ${pexp} with script interpretor path
On Mon, Sep 16, 2013 at 08:57:11PM +0200, Antoine Jacoutot wrote: Any other thoughts? Is there any way we could use something else than file(1) ? Well, you can look for a script using dd and matching it against the two possible signatures. e.g., dd 2/dev/null if=file bs=4 count=1|grep -qe '#!/./ -e '#! /' I'm not sure that's too much of a progress. There might be another way to grab the first 4 bytes of a file. Off-hand I don't see any.
Re: /etc/rc.d/rc.subr; prefix ${pexp} with script interpretor path
On Mon, Sep 16, 2013 at 11:28:06PM +0200, Alexander Hall wrote: sed can do it all. Really. Notes: - I separate re_quote() cause I think it can be useful in other places. - I think re_quote() is (basic) regex complete. - I don't care if the interpreter is (or seems) nonexistant, as that shouldn't be a runtime error. - I'm sure sed may die horribly if you try to feed it a 9GB oneline file. However, if so, it should not produce any output anyway. ;) If this would ever be considered a real problem, dd(1) would help (as espie already mentioned). re_quote() { sed 's/\([]^$*.\\[]\)/\\\1/g'; } interpreter=$( sed -n 's/^#![[:space:]]*\(.*\)/\1 /p;q' ${daemon} | re_quote) pexp=$interpreter$pexp Ah, but what happens when you run that thru a pure binary file that spans a single line ?...
Re: SQLite 3.8.0.2 diff
On Thu, Sep 12, 2013 at 02:35:02PM -0400, James Turner wrote: Attached is a diff to update our in tree version of SQLite to the recently released 3.8.0.2. SQLite 3.8.0 is needed for a fossil update I'm working on. I've tested this diff against my fossil update and everything appears to be good, the recent arc4random addition should have been maintained across the diff. This should probably go through a round of bulk builds. I'll express my thanks now for anyone who can assist by running a bulk build on a couple platforms. I'm also okay with this.
HEADS UP: tmux protocol change
be sure to not have any old sessions hanging around before you rebuild source, as you will be unable to reattach these. (quick fix: recompile tmux from -D2013-09-01, don't install the binary, but use it to attach to your old session. Just saved my ass).
config: make reconfig target
The following patch expands config(8) slightly to be able to say make reconfig from the build directory. Yeah, it's sugar, but it got tiresome having to change directories all the time. I've put the reconfig target at end, where I'm sure it can't be the first target. An alternative approach would be to add an extra keyword for boilerplate stuff that would always be emitted by config, but I don't see the point of the extra knob in that particular case. Index: config.h === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/config.h,v retrieving revision 1.25 diff -u -p -r1.25 config.h --- config.h5 Dec 2012 23:20:26 - 1.25 +++ config.h28 Oct 2013 07:44:56 - @@ -339,6 +339,10 @@ void defoption(const char *name); intdevbase_has_instances(struct devbase *, int); intdeva_has_instances(struct deva *, int); void setupdirs(void); +intpflag; +char *sflag; +char *bflag; +char *startdir; /* mkheaders.c */ intmkheaders(void); Index: main.c === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/main.c,v retrieving revision 1.44 diff -u -p -r1.44 main.c --- main.c 22 Jun 2012 22:02:29 - 1.44 +++ main.c 28 Oct 2013 07:47:13 - @@ -93,15 +93,24 @@ usage(void) exit(1); } +int pflag = 0; +char *sflag = NULL; +char *bflag = NULL; +char *startdir; + int main(int argc, char *argv[]) { char *p; const char *last_component; char *outfile = NULL; - int pflag, ch, eflag, uflag, fflag; + int ch, eflag, uflag, fflag; + char dirbuffer[PATH_MAX]; pflag = eflag = uflag = fflag = 0; + startdir = getcwd(dirbuffer, sizeof dirbuffer); + if (startdir == NULL) + warn(Can't getcwd, no make reconfig); while ((ch = getopt(argc, argv, egpfb:s:o:u)) != -1) { switch (ch) { @@ -148,10 +157,12 @@ main(int argc, char *argv[]) break; case 'b': + bflag = optarg; builddir = optarg; break; case 's': + sflag = optarg; srcdir = optarg; break; Index: mkmakefile.c === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/mkmakefile.c,v retrieving revision 1.37 diff -u -p -r1.37 mkmakefile.c --- mkmakefile.c17 Sep 2012 17:36:13 - 1.37 +++ mkmakefile.c28 Oct 2013 07:47:03 - @@ -60,6 +60,7 @@ static const char *srcpath(struct files *); static int emitdefs(FILE *); +static int emitreconfig(FILE *); static int emitfiles(FILE *, int); static int emitobjs(FILE *); @@ -120,6 +121,10 @@ mkmakefile(void) if ((*fn)(ofp)) goto wrerror; } + if (startdir != NULL) { + if (emitreconfig(ofp) != 0) + goto wrerror; + } if (ferror(ifp)) { (void)fprintf(stderr, config: error reading %s (at line %d): %s\n, @@ -271,6 +276,33 @@ emitdefs(FILE *fp) for (nv = mkoptions; nv != NULL; nv = nv-nv_next) if (fprintf(fp, %s=%s\n, nv-nv_name, nv-nv_str) 0) return (1); + return (0); +} + +static int +emitreconfig(FILE *fp) +{ + if (fputs(\n + .PHONY: reconfig\n + reconfig:\n, fp) 0) + return (1); + if (fprintf(fp, \tcd %s config , startdir) 0) + return (1); + if (pflag) { + if (fputs(-p , fp) 0) + return (1); + } + if (sflag) { + if (fprintf(fp, -s %s , sflag) 0) + return (1); + } + if (bflag) { + if (fprintf(fp, -b %s , bflag) 0) + return (1); + } + /* other options */ + if (fprintf(fp, %s\n, conffile) 0) + return (1); return (0); }
config(8) 'make config' v2
Theo says no need for reconfig, make config will do just fine. okays ? Index: config.h === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/config.h,v retrieving revision 1.25 diff -u -p -r1.25 config.h --- config.h5 Dec 2012 23:20:26 - 1.25 +++ config.h28 Oct 2013 07:44:56 - @@ -339,6 +339,10 @@ void defoption(const char *name); intdevbase_has_instances(struct devbase *, int); intdeva_has_instances(struct deva *, int); void setupdirs(void); +intpflag; +char *sflag; +char *bflag; +char *startdir; /* mkheaders.c */ intmkheaders(void); Index: main.c === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/main.c,v retrieving revision 1.44 diff -u -p -r1.44 main.c --- main.c 22 Jun 2012 22:02:29 - 1.44 +++ main.c 28 Oct 2013 07:47:13 - @@ -93,15 +93,24 @@ usage(void) exit(1); } +int pflag = 0; +char *sflag = NULL; +char *bflag = NULL; +char *startdir; + int main(int argc, char *argv[]) { char *p; const char *last_component; char *outfile = NULL; - int pflag, ch, eflag, uflag, fflag; + int ch, eflag, uflag, fflag; + char dirbuffer[PATH_MAX]; pflag = eflag = uflag = fflag = 0; + startdir = getcwd(dirbuffer, sizeof dirbuffer); + if (startdir == NULL) + warn(Can't getcwd, no make reconfig); while ((ch = getopt(argc, argv, egpfb:s:o:u)) != -1) { switch (ch) { @@ -148,10 +157,12 @@ main(int argc, char *argv[]) break; case 'b': + bflag = optarg; builddir = optarg; break; case 's': + sflag = optarg; srcdir = optarg; break; Index: mkmakefile.c === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/mkmakefile.c,v retrieving revision 1.37 diff -u -p -r1.37 mkmakefile.c --- mkmakefile.c17 Sep 2012 17:36:13 - 1.37 +++ mkmakefile.c28 Oct 2013 17:57:38 - @@ -60,6 +60,7 @@ static const char *srcpath(struct files *); static int emitdefs(FILE *); +static int emitreconfig(FILE *); static int emitfiles(FILE *, int); static int emitobjs(FILE *); @@ -120,6 +121,10 @@ mkmakefile(void) if ((*fn)(ofp)) goto wrerror; } + if (startdir != NULL) { + if (emitreconfig(ofp) != 0) + goto wrerror; + } if (ferror(ifp)) { (void)fprintf(stderr, config: error reading %s (at line %d): %s\n, @@ -271,6 +276,33 @@ emitdefs(FILE *fp) for (nv = mkoptions; nv != NULL; nv = nv-nv_next) if (fprintf(fp, %s=%s\n, nv-nv_name, nv-nv_str) 0) return (1); + return (0); +} + +static int +emitreconfig(FILE *fp) +{ + if (fputs(\n + .PHONY: config\n + config:\n, fp) 0) + return (1); + if (fprintf(fp, \tcd %s config , startdir) 0) + return (1); + if (pflag) { + if (fputs(-p , fp) 0) + return (1); + } + if (sflag) { + if (fprintf(fp, -s %s , sflag) 0) + return (1); + } + if (bflag) { + if (fprintf(fp, -b %s , bflag) 0) + return (1); + } + /* other options */ + if (fprintf(fp, %s\n, conffile) 0) + return (1); return (0); }
V3 Re: config(8) 'make config' v2
Backforth with Theo. A bit more logic. Most trees are moveable, and it's possible to do so when -b is not in use... Index: config.h === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/config.h,v retrieving revision 1.25 diff -u -p -r1.25 config.h --- config.h5 Dec 2012 23:20:26 - 1.25 +++ config.h28 Oct 2013 07:44:56 - @@ -339,6 +339,10 @@ void defoption(const char *name); intdevbase_has_instances(struct devbase *, int); intdeva_has_instances(struct deva *, int); void setupdirs(void); +intpflag; +char *sflag; +char *bflag; +char *startdir; /* mkheaders.c */ intmkheaders(void); Index: main.c === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/main.c,v retrieving revision 1.44 diff -u -p -r1.44 main.c --- main.c 22 Jun 2012 22:02:29 - 1.44 +++ main.c 28 Oct 2013 18:36:33 - @@ -93,13 +93,19 @@ usage(void) exit(1); } +int pflag = 0; +char *sflag = NULL; +char *bflag = NULL; +char *startdir; + int main(int argc, char *argv[]) { char *p; const char *last_component; char *outfile = NULL; - int pflag, ch, eflag, uflag, fflag; + int ch, eflag, uflag, fflag; + char dirbuffer[PATH_MAX]; pflag = eflag = uflag = fflag = 0; while ((ch = getopt(argc, argv, egpfb:s:o:u)) != -1) { @@ -148,10 +154,12 @@ main(int argc, char *argv[]) break; case 'b': + bflag = optarg; builddir = optarg; break; case 's': + sflag = optarg; srcdir = optarg; break; @@ -163,7 +171,15 @@ main(int argc, char *argv[]) argc -= optind; argv += optind; if (argc 1 || (eflag argv[0] == NULL)) + usage(); + if (bflag) { + startdir = getcwd(dirbuffer, sizeof dirbuffer); + if (startdir == NULL) + warn(Use of -b and can't getcwd, no make config); + } else { + startdir = ../../conf; + } if (eflag) { #ifdef MAKE_BOOTSTRAP Index: mkmakefile.c === RCS file: /build/data/openbsd/cvs/src/usr.sbin/config/mkmakefile.c,v retrieving revision 1.37 diff -u -p -r1.37 mkmakefile.c --- mkmakefile.c17 Sep 2012 17:36:13 - 1.37 +++ mkmakefile.c28 Oct 2013 18:36:46 - @@ -60,6 +60,7 @@ static const char *srcpath(struct files *); static int emitdefs(FILE *); +static int emitreconfig(FILE *); static int emitfiles(FILE *, int); static int emitobjs(FILE *); @@ -120,6 +121,10 @@ mkmakefile(void) if ((*fn)(ofp)) goto wrerror; } + if (startdir != NULL) { + if (emitreconfig(ofp) != 0) + goto wrerror; + } if (ferror(ifp)) { (void)fprintf(stderr, config: error reading %s (at line %d): %s\n, @@ -271,6 +276,33 @@ emitdefs(FILE *fp) for (nv = mkoptions; nv != NULL; nv = nv-nv_next) if (fprintf(fp, %s=%s\n, nv-nv_name, nv-nv_str) 0) return (1); + return (0); +} + +static int +emitreconfig(FILE *fp) +{ + if (fputs(\n + .PHONY: config\n + config:\n, fp) 0) + return (1); + if (fprintf(fp, \tcd %s config , startdir) 0) + return (1); + if (pflag) { + if (fputs(-p , fp) 0) + return (1); + } + if (sflag) { + if (fprintf(fp, -s %s , sflag) 0) + return (1); + } + if (bflag) { + if (fprintf(fp, -b %s , bflag) 0) + return (1); + } + /* other options */ + if (fprintf(fp, %s\n, conffile) 0) + return (1); return (0); }
Re: update perl Module::Build in base
On Fri, Nov 01, 2013 at 04:56:04PM +0100, Ingo Schwarze wrote: Oh well. To paraphase (forgive me if i misunderstand), It ought to be easy to write something better than Module::Build, and even though nobody has so far volunteered to propose an actual design, and even less to write some code, we already have a definite time schedule for retiring Module::Build. WT...? However, whetever thay do, we are certainly able to cope, either just keeping Module::Build in base or moving it to a port, whatever will be more convenient when the time comes. Go read Chromatics blog http://modernperlbooks.com/mt/index.html perl has been schizoid between practicality and moving forward for a few years now. One thing it has going for it is that there are very bright people who understand most of the issues involved, and so I'm not too worried about what's going to happen...
Re: CVS: cvs.openbsd.org: src
On Tue, Nov 19, 2013 at 12:11:17AM -0500, Ted Unangst wrote: On Mon, Nov 18, 2013 at 18:49, Philip Guenther wrote: btw, no library version change because the function stubs already existed. Hmm, since this is actually offering new functionality (by sem_open() and friends no longer always failing), I think it a minor bump would be appropriate. Consider that a program with an autoconf test of sem_open() will now return a different answer, just as if sem_open() was completely new. no? I hear you, but disagree. We fix program disabling bugs in libraries frequently without bumping. I have always thought of library versioning being more about program integrity, as in all the pieces you expect to find are all there, but it doesn't say anything about the inner workings of the pieces. As theo says, there are other library bumps later, but you're wrong. Use-case: new packages, slightly older snapshots. New packages actually make use of sem_open, because of said added functionality. Without a bump, pkg_add will allow to add them, and they won't work, because the functionality wasn't there... It is added functionality. It's not a minor bugfix.
Re: mention dpb in release(8)
On Tue, Nov 26, 2013 at 02:38:43PM +, Stuart Henderson wrote: Is this OK, or should we remove bits about building packages individually too? I'd rather send people to ports(7) directly... since that explains both single port and bulk building. Index: share/man/man8/release.8 === RCS file: /cvs/src/share/man/man8/release.8,v retrieving revision 1.66 diff -u -p -u -1 -5 -r1.66 release.8 --- share/man/man8/release.8 2 Sep 2012 13:47:25 - 1.66 +++ share/man/man8/release.8 26 Nov 2013 14:36:59 - @@ -338,25 +338,29 @@ The subsystem of contributed applications is described in .Xr ports 7 . For ease of installation ports can be pre-compiled into .Sq packages which can then be installed on multiple machines using .Xr pkg_add 1 . Packages are created by selecting an application to build (we'll call this one CATEGORY/PORT) and then running the following: as root: .Bd -literal -offset indent $ cd /usr/ports/CATEGORY/PORT $ su # make package .Ed .Pp -That's all there is to it. +To build a larger set of packages, +.Xr dpb 1 +has various facilities to manage the build, +either on a single machine or a cluster. .Sh SEE ALSO .Xr cvs 1 , +.Xr dpb 1 , .Xr pkg_add 1 , .Xr ports 7 , .Xr sudo 8 , .Xr sysmerge 8 .Sh HISTORY This document first appeared in .Ox 2.8 .
Re: mention dpb in release(8)
On Tue, Nov 26, 2013 at 03:38:49PM +, Stuart Henderson wrote: On 2013/11/26 16:28, Marc Espie wrote: On Tue, Nov 26, 2013 at 02:38:43PM +, Stuart Henderson wrote: Is this OK, or should we remove bits about building packages individually too? I'd rather send people to ports(7) directly... since that explains both single port and bulk building. Something like this? Includes a bonus diff for ports(7), the mirror-maker target was removed. Index: man8/release.8 === RCS file: /cvs/src/share/man/man8/release.8,v retrieving revision 1.66 diff -u -p -r1.66 release.8 --- man8/release.82 Sep 2012 13:47:25 - 1.66 +++ man8/release.826 Nov 2013 15:37:29 - @@ -335,22 +335,11 @@ in your release directory. .Ss 7. Make the third party packages The .Sq ports -subsystem of contributed applications is described in -.Xr ports 7 . -For ease of installation ports can be pre-compiled into +subsystem of contributed applications is capable of producing .Sq packages -which can then be installed on multiple machines using -.Xr pkg_add 1 . -Packages are created by selecting an application to build -(we'll call this one CATEGORY/PORT) and then running the following: -as root: -.Bd -literal -offset indent -$ cd /usr/ports/CATEGORY/PORT -$ su -# make package -.Ed -.Pp -That's all there is to it. +for installation, either individually or in bulk. +This is described in +.Xr ports 7 . .Sh SEE ALSO .Xr cvs 1 , .Xr pkg_add 1 , Index: man7/ports.7 === RCS file: /cvs/src/share/man/man7/ports.7,v retrieving revision 1.96 diff -u -p -r1.96 ports.7 --- man7/ports.7 11 Nov 2013 21:14:29 - 1.96 +++ man7/ports.7 26 Nov 2013 15:37:29 - @@ -83,9 +83,6 @@ offers a few useful targets. .It Ar index rebuild the ports complete index, .Pa /usr/ports/INDEX -.It Ar mirror-maker -see -.Xr mirroring-ports 7 , .It Ar pkglocatedb build a .Xr pkg_mklocatedb 1 @@ -740,6 +737,7 @@ List of users and groups created by port .Xr pkg_info 1 , .Xr bsd.port.mk 5 , .Xr port-modules 5 , +.Xr mirroring-ports 7 , .Xr packages 7 .Pp The okay for me.
Re: perlre(1) and substitution evaluations
On Sat, Nov 30, 2013 at 12:59:54PM +0200, Lars Nooden wrote: perlre(1) seems to be missing information about substitution evaluations with the /e option. The functionality is present in perl: perl -e '$_=2; s/2/1+3/e; print' But it is not listed in the base documentation. The modifier /e is described in Programming Perl, 4th ed, pp 186, 254-255. Could something like the following be added? Regards, /Lars ? pod2htmd.tmp Index: perlre.pod === RCS file: /cvs/src/gnu/usr.bin/perl/pod/perlre.pod,v retrieving revision 1.16 diff -u -p -u -p -r1.16 perlre.pod --- perlre.pod25 Mar 2013 20:40:59 - 1.16 +++ perlre.pod30 Nov 2013 10:58:38 - @@ -105,6 +105,12 @@ X/p Xregex, preserve Xregexp, prese Preserve the string matched such that ${^PREMATCH}, ${^MATCH}, and ${^POSTMATCH} are available for use after matching. +=item e +X/e + +Treat the replacement portion as an interpreted expression. +Each additional C/e modifier after the first functions as an eval() around the code to execute. + =item g and c X/g X/c For this kind of thing, you're much better off talking to upstream as this is totally openbsd-independent.
Re: Remove 4.3BSD tty compatibility, take 2
On Tue, Dec 10, 2013 at 10:31:31PM +0100, Mark Kettenis wrote: Date: Tue, 10 Dec 2013 21:48:41 +0100 From: Christian Weisgerber na...@mips.inka.de New attempt, now that the userland has been cleaned up. This diff kills the remaining parts of the COMPAT_43 tty handling in the kernel. (There are some further network-related COMPAT_43 fragments that are not touched by this.) Comments? ok? Noticed TIOCGSID in that list. Don't think that is a 4.3 compat ioctl. Rather a System V compat ioctl. Might be useful if we ever find the need to implement tcgetsid(3). There is code in gnulib (the GNU portability library) that uses this ioctl to implement tcgetsid(3): https://github.com/gagern/gnulib/blob/master/lib/tcgetsid.c Not sure if there actually is anything out there that uses it. Note that tcgetsid(3) is part of posix 2008... I don't see this as a noxious interface, so I don't see any reason not to keep it... In ports, devel/ddd uses it as a fallback on systems without setsid. lang/sbcl checks for it, but it definitely is just wanting to create a wrapper for it. (apparently, vala has similar code, just looking thru google). Oh, hey, and it's in Stevens... so we MUST have it :)
Re: Remove 4.3BSD tty compatibility, take 2
On Tue, Dec 10, 2013 at 11:05:01PM +0100, Marc Espie wrote: Note that tcgetsid(3) is part of posix 2008... I don't see this as a noxious interface, so I don't see any reason not to keep it... My bad, it's XSI in posix 2008, but I was reading posix 2013, which has: Issue 7 The tcgetsid() function is moved from the XSI option to the Base.
more predictable package installations
I'm thinking I'd like to impose a PATH for running @exec/@unexec style lines in packing-lists. Some people have their default path ignoring /usr/local, which is just fine, but the packages expect things to run in a correct way, so currently any local command requires /usr/local/bin/cmd to be run (and does that). setting PATH explicitly to /bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:${LOCALBASE}/bin:${LOCALBASE}/sbin before running @exec/@unexec would allow for reproducibility in package installations, and simplify some of the command lines.
tedu old PKG_PREFIX
I think I put this in for compatibility with the old tools years ago. I don't think anyone still uses this, so this can probably die. (I have run a grep thru the ports tree and the man pages, no reference at all) okay ? Index: OpenBSD/Delete.pm === RCS file: /build/data/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/Delete.pm,v retrieving revision 1.122 diff -u -p -r1.122 Delete.pm --- OpenBSD/Delete.pm 30 Dec 2013 09:02:37 - 1.122 +++ OpenBSD/Delete.pm 31 Dec 2013 12:12:52 - @@ -147,7 +147,6 @@ sub delete_plist my $pkgname = $plist-pkgname; $state-{pkgname} = $pkgname; - $ENV{'PKG_PREFIX'} = $plist-localbase; if (!$state-{size_only}) { $plist-register_manpage($state); manpages_unindex($state); Index: OpenBSD/PkgAdd.pm === RCS file: /build/data/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/PkgAdd.pm,v retrieving revision 1.41 diff -u -p -r1.41 PkgAdd.pm --- OpenBSD/PkgAdd.pm 31 Dec 2013 11:24:55 - 1.41 +++ OpenBSD/PkgAdd.pm 31 Dec 2013 12:13:07 - @@ -1,7 +1,7 @@ #! /usr/bin/perl # ex:ts=8 sw=4: -# $OpenBSD: PkgAdd.pm,v 1.41 2013/12/31 11:24:55 espie Exp $ +# $OpenBSD: PkgAdd.pm,v 1.40 2013/12/25 14:38:15 espie Exp $ # # Copyright (c) 2003-2010 Marc Espie es...@openbsd.org # @@ -134,15 +134,11 @@ sub handle_options $state-{do_faked} = 1; } elsif (defined $state-opt('B')) { $state-{destdir} = $state-opt('B'); - } elsif (defined $ENV{'PKG_PREFIX'}) { - $state-{destdir} = $ENV{'PKG_PREFIX'}; } if (defined $state-{destdir}) { $state-{destdir}.='/'; - $ENV{'PKG_DESTDIR'} = $state-{destdir}; } else { $state-{destdir} = ''; - delete $ENV{'PKG_DESTDIR'}; } @@ -729,8 +729,6 @@ sub really_add # $replacing = 1; # } $state-{replacing} = $replacing; - - $ENV{'PKG_PREFIX'} = $state-{localbase}; my $handler = sub { $state-{interrupted} = shift; Index: OpenBSD/PkgCheck.pm === RCS file: /build/data/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/PkgCheck.pm,v retrieving revision 1.42 diff -u -p -r1.42 PkgCheck.pm --- OpenBSD/PkgCheck.pm 25 Dec 2013 14:38:15 - 1.42 +++ OpenBSD/PkgCheck.pm 31 Dec 2013 12:13:16 - @@ -316,15 +316,11 @@ sub handle_options $self-{quick} = $self-opt('q'); if (defined $self-opt('B')) { $self-{destdir} = $self-opt('B'); - } elsif (defined $ENV{'PKG_PREFIX'}) { - $self-{destdir} = $ENV{'PKG_PREFIX'}; - } + } if (defined $self-{destdir}) { $self-{destdir} .= '/'; - $ENV{'PKG_DESTDIR'} = $self-{destdir}; } else { $self-{destdir} = ''; - delete $ENV{'PKG_DESTDIR'}; } } Index: OpenBSD/PkgCreate.pm === RCS file: /build/data/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/PkgCreate.pm,v retrieving revision 1.76 diff -u -p -r1.76 PkgCreate.pm --- OpenBSD/PkgCreate.pm31 Dec 2013 11:21:10 - 1.76 +++ OpenBSD/PkgCreate.pm31 Dec 2013 12:13:24 - @@ -198,9 +198,7 @@ sub handle_options my $base = '/'; if (defined $state-opt('B')) { $base = $state-opt('B'); - } elsif (defined $ENV{'PKG_PREFIX'}) { - $base = $ENV{'PKG_PREFIX'}; - } + } $state-{base} = $base;
Re: redo signfify options
Something like this for the program itself: Index: signify.c === RCS file: /build/data/openbsd/cvs/src/usr.bin/signify/signify.c,v retrieving revision 1.5 diff -u -p -r1.5 signify.c --- signify.c 31 Dec 2013 17:33:17 - 1.5 +++ signify.c 1 Jan 2014 15:50:11 - @@ -64,8 +64,8 @@ extern char *__progname; static void usage(void) { - fprintf(stderr, usage: %s [-N] [-I input] [-O output] [-P pubkey] [-S seckey] - -V generate | sign | verify\n, __progname); + fprintf(stderr, usage: %s -G | -S | -V [-n] [-o output] + [-p pubkey] [-s seckey] [file...]\n, __progname); exit(1); } @@ -316,64 +316,91 @@ verify(const char *pubkeyfile, const cha int main(int argc, char **argv) { - const char *verb = NULL; - const char *pubkeyfile = NULL, *seckeyfile = NULL, *inputfile = NULL, - *sigfile = NULL; - char sigfilebuf[1024]; - int ch, rounds; + const char *pubkeyfile = NULL, *seckeyfile = NULL, + *sigfilefmt = %s.sig; + int ch, rounds, needfmt, i; + enum { + NONE, + GENERATE, + SIGN, + VERIFY + } verb = NONE; + rounds = 42; - while ((ch = getopt(argc, argv, I:NO:P:S:V:)) != -1) { + while ((ch = getopt(argc, argv, GSVi:no:p:s:)) != -1) { switch (ch) { - case 'I': - inputfile = optarg; + case 'G': + if (verb) + usage(); + verb = GENERATE; + break; + case 'S': + if (verb) + usage(); + verb = SIGN; + break; + case 'V': + if (verb) + usage(); + verb = VERIFY; break; - case 'N': + case 'n': rounds = 0; break; - case 'O': - sigfile = optarg; + case 'o': + sigfilefmt = optarg; break; - case 'P': + case 'p': pubkeyfile = optarg; break; - case 'S': + case 's': seckeyfile = optarg; break; - case 'V': - verb = optarg; - break; default: usage(); break; } } argc -= optind; - if (argc != 0 || verb == NULL) - usage(); - - if (inputfile !sigfile) { - if (snprintf(sigfilebuf, sizeof(sigfilebuf), %s.sig, - inputfile) = sizeof(sigfilebuf)) - errx(1, path too long); - sigfile = sigfilebuf; - } - if (streq(verb, generate)) { - if (!pubkeyfile || !seckeyfile) + if (verb == GENERATE) { + if (argc != 0 || !pubkeyfile || !seckeyfile) usage(); generate(pubkeyfile, seckeyfile, rounds); - } else if (streq(verb, sign)) { - if (!seckeyfile || !inputfile) - usage(); - sign(seckeyfile, inputfile, sigfile); - } else if (streq(verb, verify)) { - if (!pubkeyfile || !inputfile) - usage(); - verify(pubkeyfile, inputfile, sigfile); - } else { + } else if (verb == NONE || argc == 0) { usage(); + } + + needfmt = strstr(sigfilefmt, %s) != NULL; + + if (!needfmt argc != 1) { + errx(1, -o file for several inputs); + } + + for (i = 0; i != argc; i++) { + char sigfilebuf[1024]; + const char *inputfile = argv[i]; + const char *sigfile = NULL; + + if (needfmt) { + if (snprintf(sigfilebuf, sizeof(sigfilebuf), + sigfilefmt, inputfile) = sizeof(sigfilebuf)) + errx(1, path too long); + sigfile = sigfilebuf; + } else { + sigfile = sigfilefmt; + } + if (verb == SIGN) { + if (!seckeyfile) + usage(); + sign(seckeyfile, inputfile, sigfile); + } else if (verb == VERIFY) { + if (!pubkeyfile || !inputfile) + usage(); + verify(pubkeyfile, inputfile, sigfile); + } } return 0; }
Re: redo signfify options
On Wed, Jan 01, 2014 at 12:53:09PM -0500, Ted Unangst wrote: On Wed, Jan 01, 2014 at 16:25, Marc Espie wrote: Now that you're fixing it, I think you've still got it wrong. Instead of signify -V -p pubkey -i input you want to be able to do signify -V -p pubkey input ... e.g., at this point, input would not need to be an option, so that you can verify several files in one go. we can do that too if we need batch verification, but I left it out of this commit. Well, at least make input not an option but an actual parameter. Otherwise, you have a command with no parms. And considering the command name, it makes little sense. As the name says, that command is used to sign or verify a file. The *object* of that command is the file. It's not an option !
some error fixes in signify
Revisiting the error messages: - pass the filenames to the low-level functions, so they can tell you what's going on. - FIX readall and writeall. The logic *is* wrong. Just because they return something != len, doesn't mean they return -1. Okay ? Index: signify.c === RCS file: /build/data/openbsd/cvs/src/usr.bin/signify/signify.c,v retrieving revision 1.6 diff -u -p -r1.6 signify.c --- signify.c 1 Jan 2014 17:50:33 - 1.6 +++ signify.c 2 Jan 2014 13:37:38 - @@ -92,10 +92,14 @@ xmalloc(size_t len) } static void -readall(int fd, void *buf, size_t len) +readall(int fd, void *buf, size_t len, const char *filename) { - if (read(fd, buf, len) != len) - err(1, read); + ssize_t x = read(fd, buf, len); + if (x == -1) { + err(1, read from %s, filename); + } else if (x != len) { + errx(1, short read from %s, filename); + } } static void @@ -108,7 +112,7 @@ readb64file(const char *filename, void * memset(b64, 0, sizeof(b64)); rv = read(fd, b64, sizeof(b64) - 1); if (rv == -1) - err(1, read in %s, filename); + err(1, read from %s, filename); for (i = 0; i rv; i++) if (b64[i] == '\n') break; @@ -137,7 +141,7 @@ readmsg(const char *filename, unsigned l if (msglen (1UL 30)) errx(1, msg too large in %s, filename); msg = xmalloc(msglen); - readall(fd, msg, msglen); + readall(fd, msg, msglen, filename); close(fd); *msglenp = msglen; @@ -145,10 +149,14 @@ readmsg(const char *filename, unsigned l } static void -writeall(int fd, const void *buf, size_t len) +writeall(int fd, const void *buf, size_t len, const char *filename) { - if (write(fd, buf, len) != len) - err(1, write); + ssize_t x = write(fd, buf, len); + if (x == -1) { + err(1, write to %s, filename); + } else if (x != len) { + errx(1, short write to %s, filename); + } } static void @@ -161,10 +169,10 @@ writeb64file(const char *filename, const fd = xopen(filename, O_CREAT|O_EXCL|O_NOFOLLOW|O_RDWR, mode); snprintf(header, sizeof(header), signify -- %s\n, comment); - writeall(fd, header, strlen(header)); + writeall(fd, header, strlen(header), filename); if ((rv = b64_ntop(buf, len, b64, sizeof(b64))) == -1) errx(1, b64 encode failed); - writeall(fd, b64, rv); + writeall(fd, b64, rv, filename); memset(b64, 0, sizeof(b64)); close(fd); }
Re: some error fixes in signify
On Thu, Jan 02, 2014 at 10:36:53AM -0500, Ted Unangst wrote: On Thu, Jan 02, 2014 at 14:40, Marc Espie wrote: Revisiting the error messages: - pass the filenames to the low-level functions, so they can tell you what's going on. - FIX readall and writeall. The logic *is* wrong. Just because they return something != len, doesn't mean they return -1. hmm. the intention was to keep the code as small as possible. a detailed dissection of every error that will never happen runs somewhat contrary to that goal. it's not like the difference between a failed write and a short write is meaningful to the user. Oh come on, that's a bug, and really poor style. You're going to call err() when there's no meaningful value in errno ? That's really a bad example, if nothing else. There's a difference between small and obnoxiously wrong. Even if it's a wee bit larger, it doesn't make it less easy to read.
signify API change and manpage
I really, really want to have an actual parameter for sign and verify, this makes more sense. Also, the set of modes of signify means it's much clearer to have separate synopsis in the manpage... Also fix some .Ar which should be .Fl while we're at it. To *keep things simple*, let's only sign a single file... ;) (note that this actually fixes the code logic, again, since you could specify an input file option, and generate a sigfile name, and even error out in cases things were too long if -G got used)... Note that I haven't changed the option order in the synopsis, though I would tend to put -G/-S/-V first, personally... Index: signify.1 === RCS file: /build/data/openbsd/cvs/src/usr.bin/signify/signify.1,v retrieving revision 1.6 diff -u -p -r1.6 signify.1 --- signify.1 1 Jan 2014 17:50:33 - 1.6 +++ signify.1 3 Jan 2014 13:47:54 - @@ -23,15 +23,25 @@ .Sh SYNOPSIS .Nm signify .Op Fl n -.Op Fl i Ar input +.Fl p Ar pubkey +.Fl s Ar seckey +.Fl G +.Nm signify .Op Fl o Ar output -.Op Fl p Ar pubkey -.Op Fl s Ar seckey -.Fl G | S | V +.Fl s Ar seckey +.Fl S +.Ar input +.Nm signify +.Op Fl o Ar output +.Fl p Ar pubkey +.Fl V +.Ar input .Sh DESCRIPTION The .Nm -utility creates and verifies cryptographic signatures. +utility creates and verifies cryptographic signatures for +an input file +.Ar input . The mode of operation is selected by the .Fl G , .Fl S , @@ -43,8 +53,6 @@ The options are as follows: .Bl -tag -width Ds .It Fl G Generate a new keypair. -.It Fl i Ar input -Input file to sign or verify. .It Fl n Do not ask for a passphrase during key generation. Otherwise, @@ -56,17 +64,17 @@ The default is .Ar input Ns .sig . .It Fl p Ar pubkey Public key produced by -.Ar G , +.Fl G , and used by -.Ar V +.Fl V to check a signature. .It Fl S Sign the input file. .It Fl s Ar seckey Secret (private) key produced by -.Ar G , +.Fl G , and used by -.Ar S +.Fl S to sign a message. .It Fl V Verify the input file and signature match. @@ -94,13 +102,13 @@ The message file is too large. .El .Sh EXAMPLES Create a new keypair: -.Dl $ signify -p newkey.pub -s newkey.sec -G +.Dl $ signify -G -p newkey.pub -s newkey.sec .Pp Sign a file, specifying a signature name: -.Dl $ signify -s key.sec -i message.txt -o msg.sig -S +.Dl $ signify -S -s key.sec -o msg.sig message.txt .Pp Verify a signature, using the default signature name: -.Dl $ signify -p key.pub -i generalsorders.txt -V +.Dl $ signify -V -p key.pub generalsorders.txt .Sh SEE ALSO .Xr cmp 1 , .Xr sha256 1 , @@ -109,4 +117,4 @@ Verify a signature, using the default si The .Nm command first appeared in -.Ox 5.5 +.Ox 5.5 . Index: signify.c === RCS file: /build/data/openbsd/cvs/src/usr.bin/signify/signify.c,v retrieving revision 1.7 diff -u -p -r1.7 signify.c --- signify.c 2 Jan 2014 16:34:02 - 1.7 +++ signify.c 3 Jan 2014 13:57:23 - @@ -64,8 +64,8 @@ extern char *__progname; static void usage(void) { - fprintf(stderr, usage: %s [-n] [-i input] [-o output] [-p pubkey] [-s seckey] - -G | -S | -V\n, __progname); + fprintf(stderr, usage: %s [-n] [-o output] [-p pubkey] [-s seckey] + -G | -S | -V [input]\n, __progname); exit(1); } @@ -338,7 +338,7 @@ main(int argc, char **argv) rounds = 42; - while ((ch = getopt(argc, argv, GSVi:no:p:s:)) != -1) { + while ((ch = getopt(argc, argv, GSVno:p:s:)) != -1) { switch (ch) { case 'G': if (verb) @@ -355,9 +355,6 @@ main(int argc, char **argv) usage(); verb = VERIFY; break; - case 'i': - inputfile = optarg; - break; case 'n': rounds = 0; break; @@ -376,21 +373,29 @@ main(int argc, char **argv) } } argc -= optind; - if (argc != 0) + argv += optind; + + if (verb == GENERATE) { + if (!pubkeyfile || !seckeyfile || argc != 0) + usage(); + generate(pubkeyfile, seckeyfile, rounds); + } else if (verb == NONE) { + usage(); + } + + if (argc != 1) usage(); - if (inputfile !sigfile) { + inputfile = argv[0]; + + if (!sigfile) { if (snprintf(sigfilebuf, sizeof(sigfilebuf), %s.sig, inputfile) = sizeof(sigfilebuf)) errx(1, path too long); sigfile = sigfilebuf; } - if (verb == GENERATE) { - if (!pubkeyfile || !seckeyfile) - usage(); - generate(pubkeyfile, seckeyfile, rounds); - } else if (verb == SIGN) { + if
redux: signify diff
Rechecked that -G was working (broken) and jmc wants identical SYNOPSIS/usage. Together with the \n for base64... Index: signify.1 === RCS file: /build/data/openbsd/cvs/src/usr.bin/signify/signify.1,v retrieving revision 1.6 diff -u -p -r1.6 signify.1 --- signify.1 1 Jan 2014 17:50:33 - 1.6 +++ signify.1 3 Jan 2014 13:47:54 - @@ -23,15 +23,25 @@ .Sh SYNOPSIS .Nm signify .Op Fl n -.Op Fl i Ar input +.Fl p Ar pubkey +.Fl s Ar seckey +.Fl G +.Nm signify .Op Fl o Ar output -.Op Fl p Ar pubkey -.Op Fl s Ar seckey -.Fl G | S | V +.Fl s Ar seckey +.Fl S +.Ar input +.Nm signify +.Op Fl o Ar output +.Fl p Ar pubkey +.Fl V +.Ar input .Sh DESCRIPTION The .Nm -utility creates and verifies cryptographic signatures. +utility creates and verifies cryptographic signatures for +an input file +.Ar input . The mode of operation is selected by the .Fl G , .Fl S , @@ -43,8 +53,6 @@ The options are as follows: .Bl -tag -width Ds .It Fl G Generate a new keypair. -.It Fl i Ar input -Input file to sign or verify. .It Fl n Do not ask for a passphrase during key generation. Otherwise, @@ -56,17 +64,17 @@ The default is .Ar input Ns .sig . .It Fl p Ar pubkey Public key produced by -.Ar G , +.Fl G , and used by -.Ar V +.Fl V to check a signature. .It Fl S Sign the input file. .It Fl s Ar seckey Secret (private) key produced by -.Ar G , +.Fl G , and used by -.Ar S +.Fl S to sign a message. .It Fl V Verify the input file and signature match. @@ -94,13 +102,13 @@ The message file is too large. .El .Sh EXAMPLES Create a new keypair: -.Dl $ signify -p newkey.pub -s newkey.sec -G +.Dl $ signify -G -p newkey.pub -s newkey.sec .Pp Sign a file, specifying a signature name: -.Dl $ signify -s key.sec -i message.txt -o msg.sig -S +.Dl $ signify -S -s key.sec -o msg.sig message.txt .Pp Verify a signature, using the default signature name: -.Dl $ signify -p key.pub -i generalsorders.txt -V +.Dl $ signify -V -p key.pub generalsorders.txt .Sh SEE ALSO .Xr cmp 1 , .Xr sha256 1 , @@ -109,4 +117,4 @@ Verify a signature, using the default si The .Nm command first appeared in -.Ox 5.5 +.Ox 5.5 . Index: signify.c === RCS file: /build/data/openbsd/cvs/src/usr.bin/signify/signify.c,v retrieving revision 1.7 diff -u -p -r1.7 signify.c --- signify.c 2 Jan 2014 16:34:02 - 1.7 +++ signify.c 3 Jan 2014 15:37:57 - @@ -64,8 +64,11 @@ extern char *__progname; static void usage(void) { - fprintf(stderr, usage: %s [-n] [-i input] [-o output] [-p pubkey] [-s seckey] - -G | -S | -V\n, __progname); + fprintf(stderr, usage: + \t%s [-n] -p pubkey -s seckey -G\n + \t%s [-o output] -s seckey -S input\n + \t%s [-o output] -p pubkey -V input\n, + __progname, __progname, __progname); exit(1); } @@ -170,8 +173,9 @@ writeb64file(const char *filename, const fd = xopen(filename, O_CREAT|O_EXCL|O_NOFOLLOW|O_RDWR, mode); snprintf(header, sizeof(header), signify -- %s\n, comment); writeall(fd, header, strlen(header), filename); - if ((rv = b64_ntop(buf, len, b64, sizeof(b64))) == -1) + if ((rv = b64_ntop(buf, len, b64, sizeof(b64)-1)) == -1) errx(1, b64 encode failed); + b64[rv++] = '\n'; writeall(fd, b64, rv, filename); memset(b64, 0, sizeof(b64)); close(fd); @@ -338,7 +342,7 @@ main(int argc, char **argv) rounds = 42; - while ((ch = getopt(argc, argv, GSVi:no:p:s:)) != -1) { + while ((ch = getopt(argc, argv, GSVno:p:s:)) != -1) { switch (ch) { case 'G': if (verb) @@ -355,9 +359,6 @@ main(int argc, char **argv) usage(); verb = VERIFY; break; - case 'i': - inputfile = optarg; - break; case 'n': rounds = 0; break; @@ -376,30 +377,37 @@ main(int argc, char **argv) } } argc -= optind; - if (argc != 0) - usage(); - - if (inputfile !sigfile) { - if (snprintf(sigfilebuf, sizeof(sigfilebuf), %s.sig, - inputfile) = sizeof(sigfilebuf)) - errx(1, path too long); - sigfile = sigfilebuf; - } + argv += optind; if (verb == GENERATE) { - if (!pubkeyfile || !seckeyfile) + if (!pubkeyfile || !seckeyfile || argc != 0) usage(); generate(pubkeyfile, seckeyfile, rounds); - } else if (verb == SIGN) { - if (!seckeyfile || !inputfile) - usage(); - sign(seckeyfile, inputfile, sigfile); - } else if
Re: redux: signify diff
On Fri, Jan 03, 2014 at 10:46:45AM -0500, Ted Unangst wrote: On Fri, Jan 03, 2014 at 16:39, Marc Espie wrote: Rechecked that -G was working (broken) and jmc wants identical SYNOPSIS/usage. Together with the \n for base64... + } else if (verb == NONE) { usage(); this is weird, mixing that into the middle. Well, do you prefer that ? Index: signify.1 === RCS file: /cvs/src/usr.bin/signify/signify.1,v retrieving revision 1.6 diff -u -p -r1.6 signify.1 --- signify.1 1 Jan 2014 17:50:33 - 1.6 +++ signify.1 3 Jan 2014 15:53:58 - @@ -23,15 +23,25 @@ .Sh SYNOPSIS .Nm signify .Op Fl n -.Op Fl i Ar input +.Fl p Ar pubkey +.Fl s Ar seckey +.Fl G +.Nm signify .Op Fl o Ar output -.Op Fl p Ar pubkey -.Op Fl s Ar seckey -.Fl G | S | V +.Fl s Ar seckey +.Fl S +.Ar input +.Nm signify +.Op Fl o Ar output +.Fl p Ar pubkey +.Fl V +.Ar input .Sh DESCRIPTION The .Nm -utility creates and verifies cryptographic signatures. +utility creates and verifies cryptographic signatures for +an input file +.Ar input . The mode of operation is selected by the .Fl G , .Fl S , @@ -40,11 +50,9 @@ or options. .Pp The options are as follows: -.Bl -tag -width Ds +.Bl -tag -width Dssoutput .It Fl G Generate a new keypair. -.It Fl i Ar input -Input file to sign or verify. .It Fl n Do not ask for a passphrase during key generation. Otherwise, @@ -56,17 +64,17 @@ The default is .Ar input Ns .sig . .It Fl p Ar pubkey Public key produced by -.Ar G , +.Fl G , and used by -.Ar V +.Fl V to check a signature. .It Fl S Sign the input file. .It Fl s Ar seckey Secret (private) key produced by -.Ar G , +.Fl G , and used by -.Ar S +.Fl S to sign a message. .It Fl V Verify the input file and signature match. @@ -94,13 +102,13 @@ The message file is too large. .El .Sh EXAMPLES Create a new keypair: -.Dl $ signify -p newkey.pub -s newkey.sec -G +.Dl $ signify -G -p newkey.pub -s newkey.sec .Pp Sign a file, specifying a signature name: -.Dl $ signify -s key.sec -i message.txt -o msg.sig -S +.Dl $ signify -S -s key.sec -o msg.sig message.txt .Pp Verify a signature, using the default signature name: -.Dl $ signify -p key.pub -i generalsorders.txt -V +.Dl $ signify -V -p key.pub generalsorders.txt .Sh SEE ALSO .Xr cmp 1 , .Xr sha256 1 , @@ -109,4 +117,4 @@ Verify a signature, using the default si The .Nm command first appeared in -.Ox 5.5 +.Ox 5.5 . Index: signify.c === RCS file: /cvs/src/usr.bin/signify/signify.c,v retrieving revision 1.8 diff -u -p -r1.8 signify.c --- signify.c 3 Jan 2014 15:42:22 - 1.8 +++ signify.c 3 Jan 2014 15:53:58 - @@ -64,8 +64,11 @@ extern char *__progname; static void usage(void) { - fprintf(stderr, usage: %s [-n] [-i input] [-o output] [-p pubkey] [-s seckey] - -G | -S | -V\n, __progname); + fprintf(stderr, usage: + \t%s [-n] -p pubkey -s seckey -G\n + \t%s [-o output] -s seckey -S input\n + \t%s [-o output] -p pubkey -V input\n, + __progname, __progname, __progname); exit(1); } @@ -339,7 +342,7 @@ main(int argc, char **argv) rounds = 42; - while ((ch = getopt(argc, argv, GSVi:no:p:s:)) != -1) { + while ((ch = getopt(argc, argv, GSVno:p:s:)) != -1) { switch (ch) { case 'G': if (verb) @@ -356,9 +359,6 @@ main(int argc, char **argv) usage(); verb = VERIFY; break; - case 'i': - inputfile = optarg; - break; case 'n': rounds = 0; break; @@ -377,30 +377,37 @@ main(int argc, char **argv) } } argc -= optind; - if (argc != 0) - usage(); - - if (inputfile !sigfile) { - if (snprintf(sigfilebuf, sizeof(sigfilebuf), %s.sig, - inputfile) = sizeof(sigfilebuf)) - errx(1, path too long); - sigfile = sigfilebuf; - } + argv += optind; if (verb == GENERATE) { - if (!pubkeyfile || !seckeyfile) + if (!pubkeyfile || !seckeyfile || argc != 0) usage(); generate(pubkeyfile, seckeyfile, rounds); - } else if (verb == SIGN) { - if (!seckeyfile || !inputfile) + } else if (verb == SIGN || verb == VERIFY) { + if (argc != 1) usage(); - sign(seckeyfile, inputfile, sigfile); - } else if (verb == VERIFY) { - if (!pubkeyfile || !inputfile) - usage(); - verify(pubkeyfile, inputfile, sigfile); + + inputfile = argv[0
properly terminate lines in signify
Looking more closely at the generated files, they don't have terminating newlines... The base64 routines will properly ignore newlines, since they match isspace(). okay ? Index: signify.c === RCS file: /build/data/openbsd/cvs/src/usr.bin/signify/signify.c,v retrieving revision 1.7 diff -u -p -r1.7 signify.c --- signify.c 2 Jan 2014 16:34:02 - 1.7 +++ signify.c 3 Jan 2014 15:27:16 - @@ -170,8 +170,9 @@ writeb64file(const char *filename, const fd = xopen(filename, O_CREAT|O_EXCL|O_NOFOLLOW|O_RDWR, mode); snprintf(header, sizeof(header), signify -- %s\n, comment); writeall(fd, header, strlen(header), filename); - if ((rv = b64_ntop(buf, len, b64, sizeof(b64))) == -1) + if ((rv = b64_ntop(buf, len, b64, sizeof(b64)-1)) == -1) errx(1, b64 encode failed); + b64[rv++] = '\n'; writeall(fd, b64, rv, filename); memset(b64, 0, sizeof(b64)); close(fd);
patch to test: simplify pkg addition
The following patch removes pure package additions in pkg_add: instead of having separate code paths for normal installs and for updates (with extract then install), everything follows the extract then install road. There are several reasons behind this patch: first, it simplifies the code, and makes it simpler to fix some details later on (in particular, I think there are now conditions where you may end up with a slightly suboptimal /var/db/pkg if you interrupt at the wrong time). Second, it should allow me to do some interesting things with extraction, namely reorder archives wrt packing-lists, which is a desireable property to speed up incremental updates. Index: OpenBSD/Add.pm === RCS file: /build/data/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/Add.pm,v retrieving revision 1.135 diff -u -p -r1.135 Add.pm --- OpenBSD/Add.pm 5 Jan 2014 10:24:30 - 1.135 +++ OpenBSD/Add.pm 5 Jan 2014 10:25:11 - @@ -102,14 +102,21 @@ sub perform_installation { my ($handle, $state) = @_; - $state-{archive} = $handle-{location}; $state-{end_faked} = 0; - $handle-{partial} //= {}; - $state-{partial} = $handle-{partial}; $state-progress-visit_with_size($handle-{plist}, 'install', $state); $handle-{location}-finish_and_close; } +sub perform_extraction +{ + my ($handle, $state) = @_; + + $handle-{partial} = {}; + $state-{partial} = $handle-{partial}; + $state-{archive} = $handle-{location}; + $state-progress-visit_with_size($handle-{plist}, 'extract', $state); +} + my $user_tagged = {}; sub extract_pkgname @@ -174,13 +181,21 @@ sub prepare_for_addition { } +sub extract +{ + my ($self, $state) = @_; + $state-{partial}-{$self} = 1; + if ($state-{interrupted}) { + die Interrupted; + } +} + sub install { my ($self, $state) = @_; if ($state-{interrupted}) { die Interrupted; } - $state-{partial}-{$self} = 1; } sub copy_info @@ -352,6 +367,7 @@ package OpenBSD::PackingElement::FileBas use OpenBSD::Error; use File::Basename; use File::Path; +use OpenBSD::Temp; sub prepare_for_addition { @@ -375,53 +391,6 @@ sub prepare_for_addition } } -sub install -{ - my ($self, $state) = @_; - $self-SUPER::install($state); - my $fullname = $self-fullname; - my $destdir = $state-{destdir}; - if ($fullname =~ m,^$state-{localbase}/share/doc/pkg-readmes/,) { - $state-{readmes}++; - } - - if ($state-{extracted_first}) { - if ($state-{not}) { - $state-say(moving tempfile - #1, - $destdir.$fullname) if $state-verbose = 5; - return; - } - File::Path::mkpath(dirname($destdir.$fullname)); - if (defined $self-{link}) { - link($destdir.$self-{link}, $destdir.$fullname); - } elsif (defined $self-{symlink}) { - symlink($self-{symlink}, $destdir.$fullname); - } else { - rename($self-{tempname}, $destdir.$fullname) or - $state-fatal(can't move #1 to #2: #3, - $self-{tempname}, $fullname, $!); - $state-say(moving #1 - #2, - $self-{tempname}, $destdir.$fullname) - if $state-verbose = 5; - undef $self-{tempname}; - } - } else { - my $file = $self-prepare_to_extract($state); - - $state-say(extracting #1, $destdir.$fullname) - if $state-verbose = 5; - if ($state-{not}) { - $state-{archive}-skip; - return; - } else { - $file-create; - $self-may_check_digest($file, $state); - - } - } - $self-set_modes($state, $destdir.$fullname); -} - sub prepare_to_extract { my ($self, $state) = @_; @@ -467,6 +436,96 @@ sub prepare_to_extract return $file; } +sub extract +{ + my ($self, $state) = @_; + + my $file = $self-prepare_to_extract($state); + + if (defined $self-{link} || defined $self-{symlink}) { + $state-{archive}-skip; + return; + } + + $self-SUPER::extract($state); + + # figure out a safe directory where to put the temp file + my $d = dirname($file-{destdir}.$file-name); + # we go back up until we find an existing directory. + # hopefully this will be on the same file system. + while (!-d $d -e _ || defined $state-{noshadow}-{$d}) { + $d = dirname($d); + } + if ($state-{not}) { + $state-say(extracting tempfile
Re: signify untrusted comments
On Tue, Jan 07, 2014 at 04:54:59PM +, Christian Weisgerber wrote: (I'm also open to reconsidering whether keys should include identifiers. Perhaps a random id created during key generation? Just enough to say you're using the wrong key.) I'm in favor. Yeah, looks like a good idea. 64 bits random number ?
Re: signify untrusted comments
On Tue, Jan 07, 2014 at 01:17:12PM -0500, Ted Unangst wrote: On Tue, Jan 07, 2014 at 16:54, Christian Weisgerber wrote: Ted Unangst t...@tedunangst.com wrote: To that end, I think the comment should be marked as untrusted, and signify should even check that it says untrusted. Hopefully this makes it a little harder to con somebody into believing the comment actually should be trusted. I think somebody who can be conned into accepting a key will not understand or think about untrusted comment. hmm. 1. people who don't trust strangers. we don't need to protect them. 2. people who do trust strangers. we can't protect them. 3. people who are initially suspicious, but have a mistaken belief that they can spot mischief. maybe they will be fooled regardless, but I don't want to provide anything that can be used as leverage against them. (I'm also open to reconsidering whether keys should include identifiers. Perhaps a random id created during key generation? Just enough to say you're using the wrong key.) I'm in favor. Here's a diff with that too. I print out the fingerprints after a mismatch, but the fingerprints are in the opaque part of the file, so I'm not sure how useful that is. At least it provides a hint to check the command line arguments. I think the fingerprint mismatch error message is too technical. errx(1, Signature failed: checked against the wrong key); looks like enough for me. Apart from that, I'm okay with that code. If you want to display fingerprints, well, you have to make it possible to display fingerprints for any key-like message. Not that it's much code, but is it really necessary ?
Re: signify embedded sigs
On Wed, Jan 08, 2014 at 03:53:32PM -0500, Ted Unangst wrote: This may or may not be useful. We can embed the message after the signature and reduce the number of files in half. It's not a lot of code. (This is how the crypto_sign API originally worked, but we've since added some metadata, so we still need to destruct and reconstruct the crypto_sign inputs and outputs.) The -e option is added. It works a lot like gzip/gunzip by default. You sign a message, creating message.txt.sig: signify -e -S message.txt You verify message.txt.sig, creating message.txt: signify -e -V message.txt Note you always specify the message name. I've cleaned up the source a bit to disambiguate between input and message. And of course, -o works as before. I'm not fond of embedded as a global variable, especially since it's only used by two functions. Can you pass it as a parameter, so that it's more explicit where it's used ?
fw_update experiments
We're currently trying out some new stuff, purely as an experiment (the files in /etc/signify all say so explicitly, don't put any confidence in them) if you experience trouble running fw_update, make sure you have a new snapshot. if you're trying to use pkg_add directly to grab/update firmwares, make sure to use -DFW_UPDATE on those.
Re: md5/sha256 -c option
On Fri, Jan 10, 2014 at 08:05:44AM +0001, Jason McIntyre wrote: On Fri, Jan 10, 2014 at 01:01:46AM -0700, Theo de Raadt wrote: well then this means the description of -c is very poor. i would look for a fix there, not in SYNOPSIS. But look closer, the synopsis is wrong: sha256 [-bpqrtx] [-c [checklist ...]] [-s string] [file ...] It is not regular. When does checklist ... stop and file ... start? No matter what, that line needs a change. At minimum, sha256 [-bcpqrtx] [-s string] [file ...] With wording changes below. right, so i support the idea of making a single synopsis as clear as possible, and improving the text. Sorry, that's not always possible. I think you're completely wrong and pig-headed about that one. (You probably have a natural bias towards full fledged text, whereas coder-type people will tend to trust the quick summary that looks like code, and won't have a problem parsing more information from there.) The main reason we have several synopsis is that those are actually related *commands* that happen to share the same *filename*. When I'm in a hurry, I check the SYNOPSIS. When I type wrong options, the first thing I see is the SYNOPSIS, not the manpage. Oh, hey, let's read the full manpage... hum, lots of text. Oh, hey, let's check what option goes with what. One-line synopsis for signify(1): signify [-neGVI] [-o sigfile] [-p pubkey] [-s seckey] [message] What's currently in there: signify -G [-n] -p pubkey -s seckey signify -I [-o sigfile] [-p pubkey] [-s seckey] signify -S [-e] [-o sigfile] -s seckey message signify -V [-e] [-o sigfile] -p pubkey message Sorry, the current version is *ways* more useful. In case options are completely separated by some kind of mode, it's much much clearer to put things as separate SYNOPSIS.
signed packages
It's probably time to talk about it. Yes, we are now distributing signed packages. A lot of people have probably noticed because there was a key mismatch on at least one batch of signed packages. Obviously, we haven't finished testing yet. Don't read too much into that. Signed packages just mean you can use an insecure medium, such as ftp, to download packages: if the key matches, it means the package hasn't been tampered with since it was signed. The cryptographic framework used to sign packages is called signify(1), mostly written by Ted Unangst, with a lot of feedback from (mostly) Theo and I. The signing framework in pkg_add/pkg_create is much older than that, if was written for x509 a few years ago, but signify(1) will probably be more robust and ways simpler. In particular, there's no chain-of-trust, so you keep complete control on the sources YOU trust. Signatures should be transparent in use: the package is opened, the packing-list signature is checked, and then files are checksummed while extracted against the packing-list embedded checksums (there are provisions to ensure any dangerous meta-data is also encoded in the packing-list as @mode/@user/@group annotations. So, barring problems, you shouldn't even notice signatures.
Re: pkg_add (pkg.conf): option to require signed packages
On Fri, Jan 17, 2014 at 09:59:03AM +0100, Sébastien Marie wrote: On Thu, Jan 16, 2014 at 10:02:22AM +, Stuart Henderson wrote: On 2014/01/16 08:53, Sébastien Marie wrote: Hi, Does it make sens to have an option to require package to be signed ? It makes more sense to just enable that by default, when we are happy with the infrastructure and usage. I saw enable by default more as long term purpose. The patch would permit to easily test it... Enable by default is trivial to do. Look around the code that says check_signature in OpenBSD/PkgAdd.pm, I'm sure you can figure out the change.
Re: signed packages
On Fri, Jan 17, 2014 at 12:09:31PM -0500, sven falempin wrote: Awesome. Â * the public one on the client openBSD Â * the private one on the builder is there a new make command in ports to sign ? like make sign ? make resign ? See signify(1), pkg_add(1), pkg_create(1), bsd.port.mk(5) (look for SIGNING_PARAMETERS). Packages can be signed during build, or later. There's no new command, pkg_create(1) is used for creating signed packages.
Re: signed packages
On Fri, Jan 17, 2014 at 06:23:53PM +0100, Marc Espie wrote: On Fri, Jan 17, 2014 at 12:09:31PM -0500, sven falempin wrote: Awesome. Â * the public one on the client openBSD Â * the private one on the builder is there a new make command in ports to sign ? like make sign ? make resign ? See signify(1), pkg_add(1), pkg_create(1), bsd.port.mk(5) (look for SIGNING_PARAMETERS). Packages can be signed during build, or later. There's no new command, pkg_create(1) is used for creating signed packages. Note that things are still WILDLY changing. I assume that by now, lots of people have noticed the signed stuff. This is still a moving target (working quite well IMO).
Re: signed packages
On Fri, Jan 17, 2014 at 12:39:49PM -0500, sven falempin wrote: i read the manuals , and well , i am still unsure, if i put SIGNER=bob in the package configuration then it will be signed with /etc/signify/bob.sec having to read 4 different manual page to get this is strange :p No, that part got simpler. Keys are currently under /etc/signify They *must* be there for the public keys. Keys for signed packages should match *pkg.sec / *pkg.pub (distinguished by function: firmware keys end in fw.sec / fw.pub) Read signify(1) to generate the keys. Say: signify -G -n -s sven-pkg.sec -p sven-pkg.pub For signing while building, just set SIGNING_PARAMETERS = -s signify -s /etc/signify/sven-pkg.sec for signing after building, do something like: cd /usr/ports/packages/${ARCH} mkdir signed pkg_create -j4 -v -s signify -s /etc/signify/sven-pkg.sec -o signed -S all That's all there is to it. If both the pubkey and privkey are present, the first signed package written out will be checked (signify keys don't carry any identity, they just have a fingerprint, so key mismatches are easy to create if you're not careful - the signed package carries a @signer sven-pkg annotation to select the correct key). pkg_add will trust keys under /etc/signify that match *pkg.pub If you really want to trust a specific key *only*, pkg_add -DSIGNER=sven-pkg ... If some booboo happened, pkg_add -Dnosig will not check sigs at all...
Re: signed packages
On Wed, Jan 22, 2014 at 01:46:33PM +0400, Loganaden Velvindron wrote: The signing framework in pkg_add/pkg_create is much older than that, if was written for x509 a few years ago, but signify(1) will probably be more robust and ways simpler. In particular, there's no chain-of-trust, so you keep complete control on the sources YOU trust. Can you please elborate more on the trusting part ? Both DNSSEC and RPKI have a root anchor that we're all supposed to trust, and your model is different. There's no chain of trust. pkg_add trusts pub keys under /etc/signify that end in *pkg.pub (respectively *fw.pub for firmwares). Put shit there - get shit out. the only way to get keys in there is: - base install, - explicitly putting keys there as root. There's nothing more automated. Keys are not certified nor anything.
Re: signed packages
A huge swath of clean-up has just hit the trees. Most specifically, now that it works, the signing-only code has been moved into a separate pkg_sign command. This is partly for documentation purpose: it's much simpler to document the parameters to that command separately, instead of as additions to pkg_create(1) proper. pkg_create(1) still retains the ability to create signed packages on the fly, if people want to create their own signed packages (not recommanded for really paranoid people, since the build process can be dirty), but signing existing packages is really a mostly independent process (the only common part is signing packing-lists) so it makes more sense as a separate command.
pkg_add tests
You should realize that the pkg tools have gone thru *major internal changes* over the last month or so. If you see weird things happening, now is a really good time to report them, before it's too late... Also, the 5.4 - 5.5 update is going to be fun: there was a kernel flag day. The proper way to update packages will be along the lines of: in 5.4: pkg_info -mq list pkg_delete * (update to 5.5) in 5.5, install new packages: pkg_add -z -l list Again, there might be issues. Now is a very good time to start looking...
Re: pkg_add tests
On Fri, Jan 24, 2014 at 11:15:21PM -0600, Shawn K. Quinn wrote: On Fri, Jan 24, 2014, at 06:50 AM, Marc Espie wrote: You should realize that the pkg tools have gone thru *major internal changes* over the last month or so. If you see weird things happening, now is a really good time to report them, before it's too late... Also, the 5.4 - 5.5 update is going to be fun: there was a kernel flag day. The proper way to update packages will be along the lines of: in 5.4: pkg_info -mq list pkg_delete * (update to 5.5) in 5.5, install new packages: pkg_add -z -l list Again, there might be issues. Now is a very good time to start looking... Do we have to do any of this if we have been following snapshots or -current? Are there any other gotchas related to these for those of us who are running snapshots or -current? I haven't had any obvious Bad Things jump up and bite me yet, want to keep it that way. (Software-related, at least; having my onboard SATA controller fall down and go boom kind of falls outside the realm of that.) Nope. You already went over the hurdle of timestamps. But if you have some 5.4 installs, or if people want to check how 5.4-5.5 goes, well, it will soon be the time to do so. Basically, the big change for them is that pkg_add now always installs the quirks package first, and it uses it to hunt for package names on fuzzy installs. e.g., the above scenario that I outlined should now get people over the 5.4 - 5.5 hurdle *while handling package renames correctly*. Obviously, this hasn't been tested too much yet... As for the usefulness of testing, for instance, I made a large booboo in an extra check in pkg_sign. Fortunately rpe@ caught it fairly early, so a fixed amd64 snap should be on its way out soon (and it was just garbled archives, so rsync should chew on that one really fast compared to the usual slooow trickle of snapshots out).
Re: your mail
On Fri, Jan 24, 2014 at 08:08:29PM -0500, Ted Unangst wrote: I generally associate negative connotations with so-called, as in the so-called free world. I wouldn't use it just to name something, as in the kernel is written in the so-called C language. so-called implies it's called this, but it's not. Two imo dubious occurrences in the install notes. It's not a so-called MBR partition; it is an MBR partition. Similarly with hppa LIF. There's one other use in loongson about initrd which seems ok. I agree with so-called having negative connotations. I think both those instances are using it intentionally, namely there are nasty surprises in some MBR blocks that are not covered by the so-called MBR standard. Probably likewise for hppa LIF...
Re: your mail
On Sat, Jan 25, 2014 at 11:04:21PM -0600, Shawn K. Quinn wrote: On Sat, Jan 25, 2014, at 08:53 AM, Marc Espie wrote: I agree with so-called having negative connotations. I think both those instances are using it intentionally, namely there are nasty surprises in some MBR blocks that are not covered by the so-called MBR standard. There's an actual MBR standard? If so, maintained by whom? Precisely. MBR is a dark art. We know we produce valid MBR blocks because machines don't blow up on us...
churn in pkg_add code
hopefully, things should be back to normal. last commit was revision 1.37 of PackageLocation.pm date: 2014/02/09 11:13:59; author: espie; state: Exp; lines: +4 -3; do a better job in wipe_info: close files and reap children. also wipe every state variable... fixes reget update_info in fringe cases (this should fix nigel@'s problems...) pkg_add memory usage is back down to reasonable levels, after two large fixes, and a heck of a lot of small changes... with tidying up and minor bug fixes along the way (well, some of them probably not so minor). Just to put things in perspective, my big testcase backed down from 480M to 280M... and in more normal cases, usage stops growing. The two large fixes were to prevent ProgressMeter/Term.pm from capturing full plists (with their sha values and all that long post the peremption date) and to limit the number of locally opened IO::Uncompress::Gunzip handles to 3 (issue was already there for fork()d gunzip, but as extra processes hanging around...). Small fixes include shrinking conflict representation, bringing full packing-lists in only AFTER solving dependencies, not storing full objects for simple shared dirs, and a lot lot of work on caching disk access better (PackageLocation/Repository cleanup fixes). Kuddoes to pirofti@ for suggesting some awesome introspection tools (Devel::Leak::Object, Devel::BackRef, Devel::Gladiator. That stuff rocks). So if you notice suspicious things and you have the current version, report them asap... there's precious little time to fix them before the release.
Re: churn in pkg_add code
On Sun, Feb 09, 2014 at 01:16:38PM +0100, Marc Espie wrote: hopefully, things should be back to normal. last commit was revision 1.37 of PackageLocation.pm date: 2014/02/09 11:13:59; author: espie; state: Exp; lines: +4 -3; do a better job in wipe_info: close files and reap children. also wipe every state variable... fixes reget update_info in fringe cases (this should fix nigel@'s problems...) And of course, I fucked something obvious up... So the good commit is revision 1.38
Re: churn in pkg_add code
On Sun, Feb 09, 2014 at 01:16:38PM +0100, Marc Espie wrote: hopefully, things should be back to normal. last commit was revision 1.37 of PackageLocation.pm date: 2014/02/09 11:13:59; author: espie; state: Exp; lines: +4 -3; do a better job in wipe_info: close files and reap children. also wipe every state variable... fixes reget update_info in fringe cases (this should fix nigel@'s problems...) And a second fix... revision 1.39 is the right one... Sending an mail to say problem fixed always works. If you don't do it, the bugs stay hidden in the woodwork :)
Re: release(8) patch
On Wed, Feb 12, 2014 at 08:16:26AM +0100, Mark Kettenis wrote: Just make sure you set $ARCH in your .profile. If you feel like it, how about a patch for /etc/skel ? It's not set by default !
Re: man.conf mandoc -Tlocale
On Sun, Feb 16, 2014 at 03:11:07PM +0100, Ingo Schwarze wrote: Few, probably, because mandoc(1) is fast enough that we run in on demand whenever possible and usually avoid preformatting anything during builds, or where we do preformat, we use groff for that, anyway. But even missing one single instance that is hiding somewhere would just be pointless disruption in a release. We can try specifically poisoning mandoc in ports land, so that you would see whenever it's invoked. Don't remember if you're familiar with that, but it's fairly trivial to do: the ports tree runs all the builds with PATH starting with ${WRKDIR}/bin you can poison mandoc very easily around line 2477 of bsd.port.mk.
Re: yacc stdc output
On Tue, Feb 18, 2014 at 02:50:53PM -0700, Todd C. Miller wrote: On Tue, 18 Feb 2014 16:38:07 -0500, Ted Unangst wrote: True, I phrased that poorly. What I'm assuming is that the code will be built by a compiler that supports the const keyword. Or in other words, if you're using a 25 year old cross compiler, I don't think it's unreasonable to expect you to dig up a 25 year old cross yacc either (or the yacc from openbsd 5.4). Who is actually using yacc to generate code for a compiler that doesn't accept const? Sudo 1.7.x still compiles on KR, mostly for the benefit of the bundled HP-UX compiler that has C89 support disabled. - todd Come on Jim, it's dead.
Re: missing ports.tar.gz in snapshot
On Thu, Mar 06, 2014 at 03:18:38PM +, Devin Ceartas wrote: Are all the ports in the packages in snapshots? I seem to remember needing to build some of the things I use to test, but maybe that has changed. There are a few ports not in snapshots, stuff that is not authorized for ftp redistribution. Accidents happen. Sometimes, stuff does not build. That's more frequent on less used architectures. Most builds on amd64 give 100%. All packages referenced in the ports tree that should be on amd64 do build.
using dpb for chroot builds
-current work, again. Doing chroot on distant hosts involves chroot, obviously... This doesn't work too well with sudo (the processes tends to be hard to kill, so if you've got to interrupt dpb, ouch). There are no exact instructions for using dpb in chroot settings. Obviously, you need a fairly complete chroot install (duh, including devices (so get rid of nosuid,nodev on that partition). For local host, put everything in the chroot, do the chroot manually, THEN build. For distant hosts, it's slightly simpler to connect as root, because then dpb can do chroot -u user /build *without* needing sudo, and hence resulting in distant makes that are trivial to ^Z/^C if the need arises... (-current gained changes to support that: initing a core runs whoami, thus it knows whether you're root on the distant machine, and if so, chroot runs directly without sudo).
HEADS UP: librt revert
kili@ just committed a revert of the librt addition in src and corresponding patches in ports. If you've built a tree with librt, you want to # rm -f /usr/lib/librt.a This lib was added to facilitate porting software, as posix asks for it. but since it's only a stub, it was only added as a static library. No-one would approve a shared library, as that would waste space. Unfortunately, libtool (ours and gnu's) don't cope well with static-only libraries. The untested commit of librt in source broke the ports tree. Specifically, programs such as x11/vlc, multimedia/xine-lib, or lang/php would no longer build (all 3 are using mutant versions of gnu libtool). It's possible further breakage would happen, but those 3 were broken. After almost a week (!), there has been exactly zero activity to fix the breakage. No-one volunteered to do the requisite patches, and well, these ports are not exactly low profile, we need a ports tree in working condition to be able to conduct other work (such as the pending update to perl, or some other clean-up work). So for now, the librt experiment got reverted. Maybe temporarily (). If people really want it in, they had better be willing to figure out how to fix the libtool breakage first...
Re: HEADS UP: librt revert
On Sun, Mar 23, 2014 at 10:46:08PM +0400, Loganaden Velvindron wrote: On Sun, Mar 23, 2014 at 10:34 PM, Marc Espie es...@nerim.net wrote: kili@ just committed a revert of the librt addition in src and corresponding patches in ports. If you've built a tree with librt, you want to # rm -f /usr/lib/librt.a This lib was added to facilitate porting software, as posix asks for it. but since it's only a stub, it was only added as a static library. No-one would approve a shared library, as that would waste space. Unfortunately, libtool (ours and gnu's) don't cope well with static-only libraries. The untested commit of librt in source broke the ports tree. Specifically, programs such as x11/vlc, multimedia/xine-lib, or lang/php would no longer build (all 3 are using mutant versions of gnu libtool). It's possible further breakage would happen, but those 3 were broken. After almost a week (!), there has been exactly zero activity to fix the breakage. No-one volunteered to do the requisite patches, and well, these ports are not exactly low profile, we need a ports tree in working condition to be able to conduct other work (such as the pending update to perl, or some other clean-up work). That's very sad. I get the impression that there aren't many active developers (?) No, it's more a question of people being busy with other things at times. There is a flurry of important activity in preparation for various things. It's one of those times when having a tree that works is important. A week is an fairly long time for the tree to stay broken, in OpenBSD terms. The revert is a pain, but much less pain than figuring out what to do to get the affected ports to build again in a clean fashion. And again, the concerned people were not available to do timely fixes. As for myself, for instance, I'm busy with perl 5.18, and there are some interesting new features (randomization of the hash function) that have exposed other interesting problems in our libtool... you can't wage a war on several fronts. The librt breakage was fairly distracting for a group of people that are working on other stuff. After a while, you just have to do something that lets the tree build.
Re: HEADS UP: librt revert
On Sun, Mar 23, 2014 at 05:05:06PM -0400, STeve Andre' wrote: On 03/23/14 14:34, Marc Espie wrote: kili@ just committed a revert of the librt addition in src and corresponding patches in ports. If you've built a tree with librt, you want to # rm -f /usr/lib/librt.a Shouldn't that be librt*a to get rid of librt_p.a too? Yep, it should. Though it's less likely to cause issues...
xucred change
I found out, after a bit of head-scratching, that you will have to make sure you're running the newly compiled mountd if you want cred-mapping to make sense. e.g., the usual build kernel, reboot, make build is not enough. You also have to kill mountd and start the new one, otherwise -maproot=root won't work.
heads-up: your ports *will break*
Got fed up of the nth report flavors are broken. The cause is always the same: for packages to build correctly, your setup *must* propagate some env variables thru sudo during the fake/packaging stage. So, I finally added up a variable that's there just to trigger an error in case your setup is not squeaky clean. Accordingly, be sure to integrate the latest sudoers modification (coming to a snapshot + sysmerge near you soon), and you get a fairly explicit clue from bsd.port.mk now... Error: sudo does not let env variables through
Re: Check syntax of Perl stuff at build time
On Wed, Apr 16, 2014 at 08:07:55PM +0400, Vadim Zhukov wrote: This is a PoC I'm currently using while hacking at our libtool. What I dream about is putting such code in /usr/share/bsd.perl.mk or such, allowing to run checks for all Perl-based tools in base.It's better to know something is broken at build time rather after install, isn't it? If idea and actual implementation is approved, I'll prepare and mail out a patch for /usr/share/bsd.perl.mk (or whatever) and actual Makefiles. Any recommendations or other comments are welcome. Maybe, this should be put in some other file or done differently... Thanks in advance. -- WBR, Vadim Zhukov Index: Makefile === RCS file: /cvs/src/usr.bin/libtool/Makefile,v retrieving revision 1.7 diff -u -p -r1.7 Makefile --- Makefile 16 Apr 2014 10:31:27 - 1.7 +++ Makefile 16 Apr 2014 15:58:30 - @@ -37,4 +37,12 @@ realinstall: ${INSTALL} ${INSTALL_COPY} -o ${BINOWN} -g ${BINGRP} -m ${BINMODE} \ ${.CURDIR}/libtool ${DESTDIR}${BINDIR}/libtool +PERLCHECK= libtool ${PACKAGES} +.for pm in libtool ${PERLCHECK} +CLEANFILES +=${pm:C@/@_@g}.perlcheck +all: ${pm:C@/@_@g}.perlcheck +${pm:C@/@_@g}.perlcheck: ${pm} + rm -f $@; perl -c ${.CURDIR}/${pm} touch $@ +.endfor + .include bsd.prog.mk Nope, doesn't always work. use needs to actually eval part of the code, and so for complicated code, -c will just check the syntax and say there are issues, because there is some code which really really needs the use to be evaluated to be syntaxicallt correct.
Re: ffs2 boot
On Thu, Apr 17, 2014 at 12:22:44PM -0500, Adam Thompson wrote: Given the single-threaded nature of much of the kernel, what applications do you run where multiple CPUs makes much of a difference to OpenBSD? You're a narrow-minded troll.
Re: openssl's *strlcy
On Fri, Apr 18, 2014 at 09:41:47PM -0400, Jacob L. Leifman wrote: I'm guessing that openssl was incorporated into OpenBSD base without prior sufficient audit by the OBSD devs because it was presumed to have better auditing / quality control upstream given its security critical nature and function. (A number of devs have commented in the past about the [lack of] code style, but I get the impression no-one expected the degree of *sloppiness* now being uncovered.) So here's a question, are there any other chunks of code that have been imported en-mass from an upstream source that could/should use an audit? especially, something that some of us non-developers might be able to assist with? No, you're mistaken. We've known for a while it was on the very dirty side, but there are obvious human reasons because of which people were reluctant to dive in. Note that it's on a par with a lot of opensource code out there, unfortunately. At some point, you have to make choices, as the amount of time and manpower we have is limited.
openssl fallout, dealing with it
Since the tree wants to move fast, here's a sketch of how we deal with ports breakage. - someone makes sure everyone is aware there's breakage. ports and src should communicate so we assert what got removed intentionally, what got removed by accident. - if the removal is not accidental, fix critical ports asap. By critical, I mean stuff such as, oh say, ruby, python, openldap, kdelibs, since that tends to take out a large chunk of the ports tree, thus effectively hiding further breakage. - iterate until most of the turds are gone.
.libs handling
At least jeremy@ and landry@ got confused about this, so I expect more people to ask. The algorithms for package updates are a bit complicated, to say the least. When you update a package with shared libraries, those can get split off into a separate package with a .lib name (which is well known). This package can be removed manually later, once a full upgrade has been done and nothing depends on it (it's not done automatically because user-compiled programs may depend on it... I intend to have a pkg_create -u mode (user) to be able to quickly register user programs so that we CAN remove them automatically). Anyways... In the meantime, those .libs* will participate in updates. In modern pkg_add, pkg_add -u will organize packages to update as updatesets: the smallest set of packages that need to be updated together because of interdependencies/conflicts. Guess what ? a .libs-foo-* will always get pulled in when you update foo, because those two packages belong to the same software and contain files that may conflict. That's why you'll see lines like .libs-foo-1.0 + foo-2.0 - foo-3.0 That's because pkg_add needs to know what's going on. What you're not going to see, though, is the creation of new .libs (unless you use -vvv). But that is necessary. That way, if you need to downgrade back from foo-2.0 to foo-1.0, pkg_add knows enough about it to be able to merge back .libs-foo-1.0 into foo-1.0 instead of complaining about a conflict and stopping. I should probably show the creation of new .libs? that might make things more simple to understand. What happened is that this algorithm evolved, it's one of those details I hadn't figured out yet when we embarked on correct safe updates...
Re: horribly slow fsck_ffs pass1 performance
On Sat, Apr 02, 2011 at 01:45:36PM -0500, Amit Kulkarni wrote: Hi, I am replying in a single email. I do a fsck once in a while, not regular. In the last 6-8 months I might have done it about 5 times. And I did it multi-user the few times I did it, but plan on doing it single user in future and I do plan to do it monthly. After seeing the messages when you fsck, it is better to do it monthly. FreeBSD which is the origin of FFS does a background fsck, and if Kirk McCusick feels so strongly I will do it too. (I remember somebody talking about having background fsck here on a openbsd list, but I forgot who it was). This is completely stupid. What do you trust more ? your file system, or fsck ? oth have bugs ! I'm sure of it ! so, if you run fsck, it's likely you're going to run into fsck bugs eventually (and trying fsck on a mounted partition was really, really stupid). Whereas, if you don't run fsck, you're going to run into fs bugs eventually. Now, consider this: the fs code is very heavily tested. People use it 24 hours a day, 365 days a year. Compared to THAT, the fsck code is very lightly tested. It's run only once in a while, when the power shuts down, or when you update your machines. What is more likely ? corrupting a perfectly sane filesystem by running fsck on it (which has MORE code paths to correct problems and is usually run on corrupted filesystems), or having an unseen bug in the fs code that affects only you and that fsck would be able to see ?
Re: horribly slow fsck_ffs pass1 performance
On Sun, Apr 10, 2011 at 11:27:41AM +, Miod Vallat wrote: Now, consider this: the fs code is very heavily tested. People use it 24 hours a day, 365 days a year. Except on leap years, of course. Those years see even more real-life testing happening! Good point. Maybe we should go to single user and run fsck in a loop on february 29th.
Re: horribly slow fsck_ffs pass1 performance
On Sun, Apr 10, 2011 at 02:40:09PM +0200, David Vasek wrote: On Sun, 10 Apr 2011, Marc Espie wrote: This is completely stupid. What do you trust more ? your file system, or fsck ? oth have bugs ! I'm sure of it ! so, if you run fsck, it's likely you're going to run into fsck bugs eventually (and trying fsck on a mounted partition was really, really stupid). There is an optional fsck -n of (mostly) mounted filesystems in /etc/daily. In what cases is this automated check intended to be used? It's code we got from BSD 4.4 lite2. This specific chunk was brought by millert@ in rev 1.17. I would venture that apart from file system hackers, or people with really flaky hardware, no-one should set that specific test...
the importance of writing documentation
Like most people, I have my pet peeves. I consider that documentation is very important to code quality. I ran into a school-book example a week ago, so I decided to share what I mean. There was this problem with dpb where some packages in a MULTI_PACKAGES show up as Errors when they don't build on a given architecture, even though they're marked as NOT_FOR_ARCHS-sub = ... I had fixed that particular way of building things by getting those to vanish when building all packages, so that the build would not error out, but dpb was still getting them as errors (and the remaining subpackages were built correclty) Looking more closely, dpb actually dumps information for all subpackages, including the non-building ones. Not including the non-building ones would not work, because the listing job would retry and retry. Nope, what was needed was the same mechanism that lets dpb ignore regular packages, that is the IGNORE variable. So, all that was needed was to set the IGNORE variable on a subpackage dependent basis, and dpb would happily trudge along. Thus, a patch was born, tested and committed that would make IGNORE subpackage dependent. I tried to make it as simple as could be: (the relevant part is setting all IGNORE-sub, then having IGNORE = ${IGNORE-sub} and testing IGNORE, plus marking IGNORE as subpackage dependent for dump-vars) Index: bsd.port.mk === RCS file: /home/openbsd/cvs/ports/infrastructure/mk/bsd.port.mk,v retrieving revision 1.1074 retrieving revision 1.1075 diff -u -p -r1.1074 -r1.1075 --- bsd.port.mk 28 Mar 2011 00:16:13 - 1.1074 +++ bsd.port.mk 3 Apr 2011 07:19:05 - 1.1075 @@ -92,11 +92,11 @@ _SHSCRIPT = sh ${PORTSDIR}/infrastructur _PERLSCRIPT = perl ${PORTSDIR}/infrastructure/bin # All variables relevant to the port's description -_ALL_VARIABLES = BUILD_DEPENDS IGNORE IS_INTERACTIVE \ +_ALL_VARIABLES = BUILD_DEPENDS IS_INTERACTIVE \ SUBPACKAGE MULTI_PACKAGES # and stuff needing to be MULTI_PACKAGE'd _ALL_VARIABLES_INDEXED = FULLPKGNAME RUN_DEPENDS LIB_DEPENDS \ - PKG_ARCH + PKG_ARCH IGNORE _ALL_VARIABLES_PER_ARCH = .if ${DPB:L:Mfetch} @@ -1320,7 +1320,6 @@ IS_INTERACTIVE = Yes # # Don't build a port if it comes with the base system. -IGNORE ?= TRY_BROKEN ?= No _IGNORE_REGRESS ?= .if defined(REGRESS_IS_INTERACTIVE) defined(BATCH) @@ -1328,44 +1327,49 @@ _IGNORE_REGRESS += has interactive test .elif !defined(REGRESS_IS_INTERACTIVE) defined(INTERACTIVE) _IGNORE_REGRESS += does not have interactive tests .endif -.if defined(IS_INTERACTIVE) defined(BATCH) -IGNORE += is an interactive port -.elif !defined(IS_INTERACTIVE) defined(INTERACTIVE) -IGNORE += is not an interactive port -.endif -.if !exists(${X11BASE}) -IGNORE += building ports requires X11 but ${X11BASE} not found -.endif -.if !defined(_ARCH_OK${SUBPACKAGE}) || ${_ARCH_OK${SUBPACKAGE}} == 0 -. if defined(ONLY_FOR_ARCHS${SUBPACKAGE}) -.if ${MACHINE_ARCH} == ${ARCH} -IGNORE += is only for ${ONLY_FOR_ARCHS${SUBPACKAGE}}, not ${MACHINE_ARCH} + +.for _s in ${MULTI_PACKAGES} +IGNORE${_s} ?= +. if defined(IS_INTERACTIVE) defined(BATCH) +IGNORE${_s} += is an interactive port +. elif !defined(IS_INTERACTIVE) defined(INTERACTIVE) +IGNORE${_s} += is not an interactive port +. endif +. if !exists(${X11BASE}) +IGNORE${_s} += building ports requires X11 but ${X11BASE} not found +. endif +. if !defined(_ARCH_OK${_s}) || ${_ARCH_OK${_s}} == 0 +.if defined(ONLY_FOR_ARCHS${_s}) +. if ${MACHINE_ARCH} == ${ARCH} +IGNORE${_s} += is only for ${ONLY_FOR_ARCHS${_s}}, not ${MACHINE_ARCH} +. else +IGNORE${_s} += is only for ${ONLY_FOR_ARCHS${_s}}, not ${MACHINE_ARCH} \(${ARCH}\) +. endif .else -IGNORE += is only for ${ONLY_FOR_ARCHS${SUBPACKAGE}}, not ${MACHINE_ARCH} \(${ARCH}\) +IGNORE${_s} += is not for ${NOT_FOR_ARCHS} .endif -. else -IGNORE += is not for ${NOT_FOR_ARCHS} . endif -.endif -.if ${SHARED_ONLY:L} == yes ${NO_SHARED_LIBS:L} == yes -IGNORE += requires shared libraries -.endif - -.if ${TRY_BROKEN:L} != yes -. if defined(BROKEN-${ARCH}) -IGNORE += is marked as broken for ${ARCH}: ${BROKEN-${ARCH}:Q} +. if ${SHARED_ONLY:L} == yes ${NO_SHARED_LIBS:L} == yes +IGNORE${_s} += requires shared libraries . endif -. if ${MACHINE_ARCH} != ${ARCH} defined(BROKEN-${MACHINE_ARCH}) -IGNORE += is marked as broken for ${MACHINE_ARCH}: ${BROKEN-${MACHINE_ARCH}:Q} + +. if ${TRY_BROKEN:L} != yes +.if defined(BROKEN-${ARCH}) +IGNORE${_s} += is marked as broken for ${ARCH}: ${BROKEN-${ARCH}:Q} +.endif +.if ${MACHINE_ARCH} != ${ARCH} defined(BROKEN-${MACHINE_ARCH}) +IGNORE${_s} += is marked as broken for ${MACHINE_ARCH}: ${BROKEN-${MACHINE_ARCH}:Q} +.endif +.if defined(BROKEN) +IGNORE${_s} += is marked as broken: ${BROKEN:Q} +.endif . endif -. if defined(BROKEN) -IGNORE += is marked as
Re: wprintf and friends
I'm happy with this as well. I think the libstdc++ is a good way to track i18n progress. When we implement enough of it, more code will activate. Even if we don't use it much directly wfscanf would be nice. My guess is that having both wfprintf and wfscanf will lead to working wide streams, for instance.
Re: aucat: suggest using -v100 ?
On Fri, May 27, 2011 at 07:01:28AM +, Jacob Meuser wrote: can we please just accept that a 100% perfect for everyone solution is not really possible? Of course not, that way lies mediocrity. ;-)
Re: En/disabling power button shutdown
On Sat, Jun 11, 2011 at 09:06:05PM +0200, gilbert.fernan...@orange.fr wrote: One knob to be able to turn it on so when you're working on a machine, and console dies, to be able to shutdown properly and avoid a fsck. One knob for the dwarfen guys that work in the IT mines, to bind the machines and rule them all.
Re: grep \words\
If we start adding this to some tools, it would start to make sense to support it at the source. Namely, conversion functions are a bit annoying, and it's always possible to fuck it up. Plus, same code twice = same bugs. Is there anything preventing us from adding non-standard flags to regcomp(3) and do it directly there ?
kill old stuff in pkg tools
INSTALL and DEINSTALL were deprecated in June 2007, and removed from pkg_create in January 2010. I think it's time to remove all traces of +MTREE_DIRS, +REQ, +INSTALL, +DEINSTALL from all commands. Index: pkg_add.1 === RCS file: /home/openbsd/cvs/src/usr.sbin/pkg_add/pkg_add.1,v retrieving revision 1.110 diff -u -p -r1.110 pkg_add.1 --- pkg_add.1 17 Aug 2011 10:48:27 - 1.110 +++ pkg_add.1 20 Aug 2011 10:11:57 - @@ -282,8 +282,6 @@ external scripts may fail. .It Ar updatedepends force update even if forward dependencies no longer match. .El -.It Fl I -If scripts exist for a given package, do not execute them. .It Fl i Switch on interactive mode. .Nm Index: pkg_delete.1 === RCS file: /home/openbsd/cvs/src/usr.sbin/pkg_add/pkg_delete.1,v retrieving revision 1.41 diff -u -p -r1.41 pkg_delete.1 --- pkg_delete.117 Aug 2011 10:48:27 - 1.41 +++ pkg_delete.120 Aug 2011 10:12:26 - @@ -92,8 +92,6 @@ uninstall even if not running as root. .It Ar scripts external scripts may fail. .El -.It Fl I -If scripts exist for a given package, do not execute them. .It Fl i Switch on interactive mode. .Nm Index: OpenBSD/Add.pm === RCS file: /home/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/Add.pm,v retrieving revision 1.124 diff -u -p -r1.124 Add.pm --- OpenBSD/Add.pm 12 Jul 2011 10:09:52 - 1.124 +++ OpenBSD/Add.pm 20 Aug 2011 10:08:36 - @@ -707,13 +707,6 @@ sub copy_info $self-fullname, $dest, $!); } -package OpenBSD::PackingElement::FINSTALL; -sub install -{ - my ($self, $state) = @_; - $self-run($state, 'PRE-INSTALL'); -} - package OpenBSD::PackingElement::FCONTENTS; sub copy_info { Index: OpenBSD/Delete.pm === RCS file: /home/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/Delete.pm,v retrieving revision 1.115 diff -u -p -r1.115 Delete.pm --- OpenBSD/Delete.pm 12 Jul 2011 10:30:29 - 1.115 +++ OpenBSD/Delete.pm 20 Aug 2011 10:09:16 - @@ -667,14 +667,6 @@ sub delete $self-mark_ldconfig_directory($state); } -package OpenBSD::PackingElement::FDEINSTALL; -sub delete -{ - my ($self, $state) = @_; - - $self-run($state, DEINSTALL); -} - package OpenBSD::PackingElement::Depend; sub copy_old_stuff { Index: OpenBSD/PackageInfo.pm === RCS file: /home/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/PackageInfo.pm,v retrieving revision 1.55 diff -u -p -r1.55 PackageInfo.pm --- OpenBSD/PackageInfo.pm 18 Aug 2011 07:53:59 - 1.55 +++ OpenBSD/PackageInfo.pm 20 Aug 2011 10:14:30 - @@ -23,8 +23,8 @@ require Exporter; our @ISA=qw(Exporter); our @EXPORT=qw(installed_packages installed_info installed_name info_names is_info_name installed_stems lock_db unlock_db -add_installed delete_installed is_installed borked_package CONTENTS COMMENT DESC INSTALL DEINSTALL REQUIRE -REQUIRED_BY REQUIRING DISPLAY UNDISPLAY MTREE_DIRS); +add_installed delete_installed is_installed borked_package CONTENTS COMMENT DESC +REQUIRED_BY REQUIRING DISPLAY UNDISPLAY); use OpenBSD::PackageName; use OpenBSD::Paths; @@ -32,21 +32,17 @@ use constant { CONTENTS = '+CONTENTS', COMMENT = '+COMMENT', DESC = '+DESC', - INSTALL = '+INSTALL', - DEINSTALL = '+DEINSTALL', - REQUIRE = '+REQUIRE', REQUIRED_BY = '+REQUIRED_BY', REQUIRING = '+REQUIRING', DISPLAY = '+DISPLAY', - UNDISPLAY = '+UNDISPLAY', - MTREE_DIRS = '+MTREE_DIRS' }; + UNDISPLAY = '+UNDISPLAY'}; use Fcntl qw/:flock/; my $pkg_db = $ENV{PKG_DBDIR} || OpenBSD::Paths-pkgdb; my ($list, $stemlist); -our @info = (CONTENTS, COMMENT, DESC, REQUIRE, INSTALL, DEINSTALL, REQUIRED_BY, REQUIRING, DISPLAY, UNDISPLAY, MTREE_DIRS); +our @info = (CONTENTS, COMMENT, DESC, REQUIRED_BY, REQUIRING, DISPLAY, UNDISPLAY); our %info = (); for my $i (@info) { Index: OpenBSD/PackingElement.pm === RCS file: /home/openbsd/cvs/src/usr.sbin/pkg_add/OpenBSD/PackingElement.pm,v retrieving revision 1.199 diff -u -p -r1.199 PackingElement.pm --- OpenBSD/PackingElement.pm 24 Jun 2011 22:43:58 - 1.199 +++ OpenBSD/PackingElement.pm 20 Aug 2011 10:10:10 - @@ -1631,35 +1631,6 @@ sub category() { OpenBSD::PackageInfo::C sub write {} -package OpenBSD::PackingElement::ScriptFile; -our @ISA=qw(OpenBSD::PackingElement::SpecialFile); -use OpenBSD::Error; - -sub exec_on_add { 1 } -sub exec_on_delete { 1 } - -sub run -{ - my ($self, $state, @args) = @_; - - my $pkgname = $state-{pkgname}; - my $name = $self-fullname; - - return if $state-{dont_run_scripts}; -
more old stuff
Apparently, nobody cares about fat packages. Not surprisingly, killing that code simplifies a few things. Especially since the necessity of passing arch around was only due to the possibility of fat packages... Index: package.5 === RCS file: /cvs/src/usr.sbin/pkg_add/package.5,v retrieving revision 1.9 diff -u -p -r1.9 package.5 --- package.5 11 Jan 2010 10:16:20 - 1.9 +++ package.5 24 Aug 2011 10:35:28 - @@ -29,7 +29,6 @@ and .Xr pkg_merge 1 , and are usually manipulated using .Xr pkg_add 1 , -.Xr pkg_merge 1 , .Xr pkg_mklocatedb 1 , or .Xr pkg_info 1 . @@ -57,9 +56,6 @@ All types of archive contents can be pre including files, directories, hardlinks, symlinks, fifos, block and character devices. .Pp -A special extension to the format, dubbed fat packages, is described in -the next section. -.Pp In order to allow just-in-time extraction, packages always begin with a table of contents, named .Pa +CONTENTS . @@ -74,8 +70,8 @@ See for annotation details. .Pp This table of contents is always followed by a few special files, some of -which are optional: the package description (+DESC), an installation script -(+INSTALL), a display message (+DISPLAY), etc. +which are optional: the package description (+DESC), +a display message (+DISPLAY), etc. .Pp The ustar format has some limitations with respect to file names. Accordingly, the package tools will replace very long names with @@ -103,38 +99,17 @@ Once the packing-list signature is check will be checksummed, resulting in a .Sq just-in-time signature checking. -.Sh FAT PACKAGES DESCRIPTION -The -.Xr pkg_merge 1 -command can create fat packages, which coalesce several normal packages in -a single ustar archive, by interleaving their contents. -.Pp -Other tools, such as -.Xr pkg_add 1 , -are aware of fat packages and can handle them transparently. .Pp -In a fat package, every item has a small prefix that identifies the -original package. -For instance, after merging two packages, the package will usually -begin with -.Pa a/+CONTENTS -and -.Pa b/+CONTENTS . -Individual items will then begin with -.Pa ab/file , -for a file common to both packages; -.Pa a/file -for a file belonging to the first package; -and -.Pa b/file -for a file belonging to the second package. +Fat packages were removed in +.Ox 5.1 , +since no practical application was found. .Sh SEE ALSO .Xr pkg_add 1 , .Xr pkg_create 1 , .Xr pkg_info 1 , -.Xr pkg_merge 1 , .Xr packages 7 , .Xr packages-specs 7 +.Sh HISTORY .Sh STANDARDS Packages are valid gzip'ed ustar archives that can be extracted using .Xr tar 1 . Index: pkg_merge === RCS file: pkg_merge diff -N pkg_merge --- pkg_merge 28 Jul 2010 12:19:54 - 1.21 +++ /dev/null 1 Jan 1970 00:00:00 - @@ -1,257 +0,0 @@ -#! /usr/bin/perl -# Copyright (c) 2005-2007 Marc Espie es...@openbsd.org -# $OpenBSD: pkg_merge,v 1.21 2010/07/28 12:19:54 espie Exp $ -# -# Permission to use, copy, modify, and distribute this software for any -# purpose with or without fee is hereby granted, provided that the above -# copyright notice and this permission notice appear in all copies. -# -# THE SOFTWARE IS PROVIDED AS IS AND THE AUTHOR DISCLAIMS ALL WARRANTIES -# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF -# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR -# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES -# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN -# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF -# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. - -use strict; -use warnings; - -use OpenBSD::PackageLocator; -use OpenBSD::PackageInfo; -use OpenBSD::PackingList; -use OpenBSD::Getopt; -use OpenBSD::Error; -use OpenBSD::Ustar; -use OpenBSD::ArcCheck; -use OpenBSD::Paths; -use OpenBSD::State; -use File::Copy; -use File::Path; - -package OpenBSD::PackingElement; -sub copy_over {} - -sub mark_tocopy {} - -sub make_alias {} -package OpenBSD::PackingElement::FileBase; -sub mark_tocopy -{ - my ($self, $list) = @_; - push(@$list, $self); -} - -sub copy_over -{ - my ($self, $wrarc, $prefix, $pkg, $state) = @_; - my $e = $pkg-{pkg}-next; - if (!$e-check_name($self)) { - $state-fatal(Names don't match: #1 vs #2, - $e-{name}, $self-{name}); - } - $e-{name} = $prefix./.$e-{name}; - $e-copy_long($wrarc); -} - -sub make_alias -{ - my ($self, $wrarc, $prefix, $pkg, $alias, $state) = @_; - my $e = $pkg-{pkg}-next; - if (!$e-check_name($self)) { - $state-fatal(Names don't match: #1 vs #2, - $e-{name}, $self-{name}); - } - $e-{name} = $prefix./.$e-{name}; - $e-alias($wrarc, $prefix/$alias); -} - -package OpenBSD
Re: more old stuff
On Wed, Aug 24, 2011 at 07:22:47PM +0200, Landry Breuil wrote: On Wed, Aug 24, 2011 at 12:39:07PM +0200, Marc Espie wrote: Apparently, nobody cares about fat packages. Not surprisingly, killing that code simplifies a few things. Especially since the necessity of passing arch around was only due to the possibility of fat packages... I don't see the relation between fat package and arch.. was one of the planned usecase 'build a single package containing a pkg for each archs' ? Read the removed code. This is most of it.
Re: diff to PackageLocator.pm to support multiple installpath entries
On Wed, Sep 07, 2011 at 12:26:30PM +0200, Marian Hettwer wrote: Hi there, see attached diff to PackageLocator.pm. It seems like in it's current version it doesn't support multiple installpath entries, like: installpath = ftp://mymirror/localrepo installpath += ftp://mymirror/officialrepo Before it was only honoring the first entry, no matter whether += or = for the first entry. However, now it works as documented in pkg.conf(5) best regards, Marian http://crivens.terrorteam.de/~rabauke/OpenBSD/PackageLocator.pm.diff or Index: usr.sbin/pkg_add/OpenBSD/PackageLocator.pm === RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackageLocator.pm,v retrieving revision 1.99 diff -p -u -r1.99 PackageLocator.pm --- usr.sbin/pkg_add/OpenBSD/PackageLocator.pm26 Aug 2011 08:46:10 - 1.99 +++ usr.sbin/pkg_add/OpenBSD/PackageLocator.pm7 Sep 2011 10:19:24 - @@ -40,7 +40,7 @@ sub build_default_path return; } $default_path-add(OpenBSD::PackageRepository-new(./, $state)-can_be_empty); - if (my $i = $state-config-value(installpath)) { + for my $i ($state-config-value(installpath)) { $default_path-add(OpenBSD::PackageRepository-new($i, $state)); } } 2 points: 1/ pkg.conf(5) does not actually say that all variables are lists. And it was not really intended for installpath (in particular, if you make installpath into a list, your next binary upgrade of the base system is going to give really funny results). 2/ the whole repository/locator/handle mess is under reconstruction, as it is a total mess, and doesn't even do what I want it to do correctly. Not exactly the right time to tweak minor bits in code that's directly headed to the trashcan...
the newer changes in the ports tree (normalization of pkgpath)
I promised yesterday I was going to write an email explaining what this is about. This has got everything to do with reproducible package builds, and the complexity that's package version numbers. First, here's the situation before the change - Take minitube-1.5, for instance. Among other things, it has RUN_DEPENDS = multimedia/gstreamer-0.10/plugins-base this ends up in the package as something like this: @depend multimedia/gstreamer-0.10/plugins-base:gstreamer-plugins-base-*:gstreamer-plugins-base-0.10.35p0 when you want to update a package, very often, the package name itself won't change. So when do we update ? pkg_add actually looks at the depend lines in the package and compares them. That is, the @depend multimedia/gstreamer-0.10/plugins-base:gstreamer-plugins-base-*: part is used as key, and the gstreamer-plugins-base-0.10.35p0 part is used for further comparison. If the set of keys change, the packages won't be comparable based on depend line, and normally, you're supposed to bump the REVISION. Otherwise, pkg_add will complain it doesn't know which is the newest package. Prior to the most recent change, the RUN_DEPENDS was used as-is. Let's look at net/avahi, it contains: RUN_DEPENDS-gtk = net/avahi,-ui,no_mono,no_qt3,no_qt4 thus, the plist for avahi-gtk-0.6.30p2 contains: @depend net/avahi,-ui,no_mono,no_qt3,no_qt4:avahi-ui-*:avahi-ui-0.6.30 However, this is somewhat strange, because the no_mono,no_qt3,no_qt4 part *is* a pseudo-flavor: it is just pertinent to the build, but not at all for a binary package. avahi-gtk would be happy with an avahi-ui that would come as @depend net/avahi,-ui,no_qt4:avahi-ui-*:avahi-ui-0.6.30 But this doesn't work, because the binary packages don't know about pseudo-flavors, and thus net/avahi,-ui,no_qt4 and net/avahi,-ui,no_mono,no_qt3,no_qt4 are completely different things, thus the packages will be different, and non comparable. The change itself - Some time ago, I came to the realization that some stuff were better left to the depending port. We always went there to get the build package name anyways (avahi-ui-0.6.30), but we also decided to get there to grab the pkgspec (the avahi-ui-* part). That's how ports depending on python magically got their python-=2.7,2.8 spec without having to say anything. For instance, youtube-dl-2011.08.04 has: @depend lang/python/2.7:python-=2.7,2.8:python-2.7.1p11 So I realized I could go to the depending port for the pkgpath *as well*. This has the neat effect of putting back the path in a standard order, and of removing pseudo-flavors entirely... it also creates full pkgpaths every time. There was some fallout: all of a sudden, *a lot* of @depend lines in packages would change, and I would need to bump all the affected packages (1980 ports to be exact). Because alongside pseudo-flavors, all multi-package dependencies would have a subpackage appended if it wasn't already there. The result -- Let's go back to our examples. minitube got bumped to minitube-1.5p0 it now sports @depend multimedia/gstreamer-0.10/plugins-base,-main:gstreamer-plugins-base-*:gstreamer-plugins-base-0.10.35p1 because of the subpackage part. youtube-dl-2011-08-04p0 sports: @depend lang/python/2.7,-main:python-=2.7,2.8:python-2.7.1p12 for the same reason. More interestingly, avahi-gtk-0.6.30p3 now says: @depend net/avahi,-ui:avahi-ui-*:avahi-ui-0.6.30p0 That's right, the pseudo-flavors are completely gone. This triggered a small bug in pkg_create (net/avahi,-ui, in that context, means the -ui subpackage of net/avahi with an empty flavor, not a default flavor) which is now fixed. Why this is cool well, the previous way pkgpaths were stored, you had to be really careful how you presented stuff, because equivalent paths would end up as different strings, and thus divergent paths stored in a package, with ill effect. For instance some/dir,flavor1,flavor2 and some/dir,flavor2,flavor1 are now completely equivalent for package building. Also, mentioning a subpackage explicitly (or not mentioning it) won't change the string recorded in the package. And finally, the new version *doesn't care at all* about pseudo-flavors. So, if you want to depend on print/poppler,no_qt,no_qt4 or print/poppler the resulting @depend line will be the same: it will be something like: @depend print/poppler,-main:poppler-*:poppler-0.6.17p1 This can solve some nightmarish situations between subpackages in the same port with pseudo-flavors, where you had to be very careful about which pseudo-flavor you were depending on if you wanted to have reproducible results. Thus removing a lot of the interest of pseudo-flavors to avoid building things (looking at such cases, the current situation is slightly wrong in a lot of directories). There's also the very real possibility to tweak pseudo-flavors in dependencies between individual builds and bulk builds thru dpb. So,
Re: tar -J for xz
On Wed, Oct 05, 2011 at 12:05:18AM +0100, Nicholas Marriott wrote: And from the look of it they also have -J, but .xz is so rarely used right now I'm not sure it is something we really need to copy... That's not gonna be the case from gnome3 (for instance). As I've been told by Antoine, they're going to drop bz2 tarballs entirely.
Re: terminal emulators using /usr/local/share/terminfo and a bug in perl's Term::Cap
On Thu, Oct 06, 2011 at 04:00:21PM +0100, Nicholas Marriott wrote: I can't really follow what it is trying to do. Won't your change always call infocmp if TERMCAP is not set instead of searching through the files? It looks like currently infocmp is only used if there are no termcap files at all, not if the entry is missing from the files it does find. So shouldn't the fallback infocmp call happen much later? Somewhere around the line: croak Can't find $term if $entry eq ''; On Thu, Oct 06, 2011 at 04:46:45PM +0200, David Coppa wrote: On Sun, Oct 2, 2011 at 8:22 AM, Nicholas Marriott nicholas.marri...@gmail.com wrote: This seems fine to me, but I'm not a perl guru. Have you talked to upstream? I'm having big difficulties trying to report the bug on rt.cpan.org. It seems they do all the best to make you feel uncomfortable :( Can this be fixed locally at least for now if my patch is correct? ciao David On Thu, Sep 29, 2011 at 10:49:36AM +0200, David Coppa wrote: Hi, The patch to ncurses nicm@ commited some days ago, exposes a bug in perl's cpan/Term-Cap/Cap.pm. So, when you use rxvt-unicode on a recent -current, you will hit this bug with pkg_* tools: # pkg_delete -v xclip failed termcap lookup on rxvt-unicode-256color at /usr/libdata/perl5/OpenBSD/ProgressMeter/Term.pm line 125 If you use a terminal emulator that has a terminfo entry but not a termcap one (just like x11/rxvt-unicode or x11/st), you should fall back to the case where perl uses infocmp to fake up a termcap entry from terminfo, but this never happens because the logic is flawed. Here's a diff: Index: cpan/Term-Cap/Cap.pm === RCS file: /cvs/src/gnu/usr.bin/perl/cpan/Term-Cap/Cap.pm,v retrieving revision 1.1.1.1 diff -u -p -r1.1.1.1 Cap.pm --- cpan/Term-Cap/Cap.pm ? ? ?24 Sep 2010 14:49:05 - ? ? ?1.1.1.1 +++ cpan/Term-Cap/Cap.pm ? ? ?29 Sep 2011 08:14:46 - @@ -273,7 +273,7 @@ sub Tgetent ? ? ?my @termcap_path = termcap_path(); - ? ?unless ( @termcap_path || $entry ) + ? ?if ( !@termcap_path || !$entry ) ? ? ?{ ? ? ? ? ?# last resort--fake up a termcap from terminfo --- cheers, David I want to test it, which I haven't had the time to do yet.
Re: enable aucat by default
On Thu, Oct 06, 2011 at 05:04:03PM -0700, patrick keshishian wrote: yuck... sounds like linux to me. please don't go down that route of enable everything, it can't hurt even if you don't use it mentality. In a wide difference to linux, we try to avoid running buggy shit on the system by default. I mean, if aucat was a large piece of junk like gnome's tracker, or firefox, I would understand objections to running it by default. Heck, even on servers. What's the actual footprint of an idle aucat ? It's not even accessible from outside, so there's no possibility of a remote hole.