Re: WANTLIB/LIB_DEPENDS semantics change
Current tests goes fine, and actually show some existing problems because the old infrastructure is: - too complicated - bsd.port.mk lies about what's going on. Namely, libspecs were apparently tied to the pkgspec in a LIB_DEPENDS. But that's not true ! bsd.port.mk was only using those specs as witness to see if the LIB_DEPENDS should be used as a RUN_DEPENDS, without ever checking that the lib came from THAT one. The new code *does* really check, for old style LIB_DEPENDS, that the libspec indeed comes from the right location. And lo and behold, I've already caught 3 ports which were attaching the wrong libspec to a given LIB_DEPENDS. So, floating the libspecs out is the right thing to do...
shared libs and ports, maybe a proposal
each time xenocara farts, we get new libs (or less libs). in order for updates to work, we *should* propagate those changes to @wantlib in each port. This currently isn't done automatically... check-lib-depends is sloooww (needs to check every file before packaging) and not even flawless. I'm beginning to think we should split *most* of the related info info a separate file, so that updates can be done automatically (for the most part). Briefly, the only info that changes is - p* numbers (e.g., PKGNAME = somethingpN) - WANTLIB for the affected package. As much as I don't like adding, as naddy would put it, more magic, I think the p* numbers could live in a separate file, along with most WANTLIB. e.g., we would have something like: REVISION-main = 5 REVISION-a = 2 WANTLIB-main += foo bar WANTLIB-a += x z in a separate file (say libinfo) (or we can even have this in the main Makefile, with proper markers for auto-update). Each time check-lib-depends runs, it would update this file... People could also update the REVISION* numbers carefully. (having WANTLIB += is because in some cases, we do need to have basic wantlib first) I'm going to see if I can make check-lib-depends perfect. I'd like a bit of input about this scheme...
Re: shared libs and ports, maybe a proposal
On Thu, Jul 08, 2010 at 12:03:47PM +0100, Stuart Henderson wrote: On 2010/07/08 11:50, Marc Espie wrote: each time xenocara farts, we get new libs (or less libs). in order for updates to work, we *should* propagate those changes to @wantlib in each port. This currently isn't done automatically... check-lib-depends is sloooww (needs to check every file before packaging) and not even flawless. I'm beginning to think we should split *most* of the related info info a separate file, so that updates can be done automatically (for the most part). Briefly, the only info that changes is - p* numbers (e.g., PKGNAME = somethingpN) - WANTLIB for the affected package. As much as I don't like adding, as naddy would put it, more magic, I think the p* numbers could live in a separate file, along with most WANTLIB. e.g., we would have something like: REVISION-main = 5 REVISION-a = 2 WANTLIB-main += foo bar WANTLIB-a += x z in a separate file (say libinfo) (or we can even have this in the main Makefile, with proper markers for auto-update). Each time check-lib-depends runs, it would update this file... People could also update the REVISION* numbers carefully. (having WANTLIB += is because in some cases, we do need to have basic wantlib first) I'm going to see if I can make check-lib-depends perfect. I'd like a bit of input about this scheme... I like having REVISION defined separately from PKGNAME, it will definitely makes it easier to update in batches, and removes a source of possible errors. I'm less keen on having this in a separate file - I see why you want to do this but I think the disadvantages outweigh it - slower checkouts, more inodes, cvs conflicts in more files, and until every existing port is moved over I think the inconsistency between different ports will be a source of confusion.. I would be happy to have this in the main Makefile with markers (and also to require certain standards for WANTLIB entries there to allow automated updates - e.g. I think requiring separate += lines rather than \ continuations is probably a good idea here). Of course we need to cater for other WANTLIB entries e.g. for the cases when a flavour adds new wantlib. I expect that specific FLAVORS will still require manual intervention. That's one reason for the WANTLIB += adding extra stuff if it's not detected is very simple. Separate file vs same file: + upstream-specific churn will show up only in that file + easier to update automatically - more inodes. - cvs conflicts (I doubt it will matter that much). I guess I can make a combination of introspection (reading existing values directly with make show=VALUE) + automatic modification (touch only a part of the Makefile that's explicitly # XXX don't touch) to put things in the same file...
Re: shared libs and ports, maybe a proposal
On Thu, Jul 08, 2010 at 08:22:38PM +, Christian Weisgerber wrote: I think both sthen@ and I have mentioned before that we really like how FreeBSD has the revision in a separate PORTREVISION variable that is much easier and less error prone to increment than pX suffixes in PKGNAME. e.g., we would have something like: REVISION-main = 5 REVISION-a = 2 Yes. And consider putting vX in a separate variable as well while you're there (` la FreeBSD's PORTEPOCH). As far as naming goes, are you okay with REVISION(-sub) and EPOCH(-sub) ? I don't see the point in adding an extra PORT in front of it. (I can do it smoothly, without needing to change every port Makefile over night).
Re: shared libs and ports, maybe a proposal
I remember why I did not do it before... there is a technical.
The way we build FULLPKGNAMEs with the interaction with flavors makes this
rather awkward.
Anyways, my make-fu did improve markedly since last time I tried, so this
ought to do the trick.
If you get a headache while reading this, you've been warned !
(note that we pass REVISION and EPOCH to pkg_create, as we do pass
flavor markers. Who knows ? might add some more checks eventually)
Index: bsd.port.mk
===
RCS file: /cvs/ports/infrastructure/mk/bsd.port.mk,v
retrieving revision 1.1017
diff -u -p -r1.1017 bsd.port.mk
--- bsd.port.mk 8 Jul 2010 20:48:39 - 1.1017
+++ bsd.port.mk 9 Jul 2010 09:34:07 -
@@ -401,9 +401,15 @@ SUBPACKAGE ?= -main
.endif
_MULTI_PACKAGES =
+
+REVISION ?=
+EPOCH ?=
+
.for _s in ${MULTI_PACKAGES}
+REVISION${_s} ?= ${REVISION}
+EPOCH${_s} ?= ${EPOCH}
-# ONLY_FOR_ARCHS is special, since it can be undefined
+# ONLY_FOR_ARCHS/NOT_FOR_ARCHS are special
. if defined(ONLY_FOR_ARCHS)
ONLY_FOR_ARCHS${_s} ?= ${ONLY_FOR_ARCHS}
. endif
@@ -542,7 +548,25 @@ _MASTER ?=
_SOLVING_DEP ?= No
_READMES =
+
.if ${MULTI_PACKAGES} == -
+# XXX parse FULLPKGNAME: is there a flavor after the version number
+.if ${FULLPKGNAME:M*-[0-9]*-*}
+. if !empty(REVISION)
+# XXX simplest way is to alter FULLPKGNAME in place, even though := is evil...
+FULLPKGNAME := ${FULLPKGNAME:C/-([0-9][^-]*)-/-\1p${REVISION}-/}
+. endif
+. if !empty(EPOCH)
+FULLPKGNAME := ${FULLPKGNAME:C/-([0-9][^-]*)-/-\1v${EPOCH}-/}
+. endif
+.else
+. if defined(REVISION)
+FULLPKGNAME := ${FULLPKGNAME}p${REVISION}
+. endif
+. if defined(EPOCH)
+FULLPKGNAME := ${FULLPKGNAME}v${EPOCH}
+. endif
+.endif
FULLPKGNAME- = ${FULLPKGNAME}
_READMES += ${READMES_TOP}/${PKGPATH}/${FULLPKGNAME}.html
.else
@@ -558,6 +582,22 @@ FULLPKGNAME${_s} = ${PKGNAME${_s}}${FLAV
FULLPKGNAME${_s} = ${PKGNAME}${_s}${FLAVOR_EXT}
. endif
.endif
+# XXX see comments above for !MULTI_PACKAGES case
+.if ${FULLPKGNAME${_s}:M*-[0-9]*-*}
+. if !empty(REVISION${_s})
+FULLPKGNAME${_s} := ${FULLPKGNAME${_s}:C/-([0-9][^-]*)-/-\1p${REVISION${_s}}-/}
+. endif
+. if !empty(EPOCH${_s})
+FULLPKGNAME${_s} := ${FULLPKGNAME${_s}:C/-([0-9][^-]*)-/-\1v${EPOCH${_s}}-/}
+. endif
+.else
+. if defined(REVISION${_s})
+FULLPKGNAME${_s} := ${FULLPKGNAME${_s}}p${REVISION${_s}}
+. endif
+. if defined(EPOCH${_s})
+FULLPKGNAME${_s} := ${FULLPKGNAME${_s}}v${EPOCH${_s}}
+. endif
+.endif
_READMES += ${READMES_TOP}/${PKGPATH}/${FULLPKGNAME${_s}}.html
. endfor
.endif
@@ -889,6 +929,12 @@ _tmpvars += ${_v}=${${_v:S/^^//}:Q}
PKG_ARGS${_S} += -DFULLPKGPATH=${FULLPKGPATH${_S}}
PKG_ARGS${_S} += -DPERMIT_PACKAGE_CDROM=${PERMIT_PACKAGE_CDROM${_S}:Q}
PKG_ARGS${_S} += -DPERMIT_PACKAGE_FTP=${PERMIT_PACKAGE_FTP${_S}:Q}
+. if !empty(REVISION${_S})
+PKG_ARGS${_S} += -DREVISION=${REVISION${_S}}
+. endif
+. if !empty(EPOCH${_S})
+PKG_ARGS${_S} += -DEPOCH=${EPOCH${_S}}
+. endif
.endfor
SUBST_CMD = perl ${PORTSDIR}/infrastructure/build/pkg_subst
RFC: changes to ports infrastructure
I want to tweak the directory structure for ports. I'd like to go for a lot of stuff to ports/infrastructure/bin ports/infrastructure/lib ports/infrastructure/man the current setup (build, fetch, install, package) is my mistake, and frankly it sucks. Some of the scripts are callable outside of bsd.port.mk, and you have to remember the path. There is no standard documentation out of the scripts, and there are a few perl modules that could use one single place. I would keep db, plist, templates. I don't think they're that confusing. The only drawback is that we're going to lose cvs history. doesn't seem like such a big problem to me. Also, third party may lose compatibility as scripts move around. That's easy to solve with symlinks, so I don't care too much. Did I miss anything ?
Re: RFC: changes to ports infrastructure
On Thu, Aug 19, 2010 at 11:09:14AM +0100, Stuart Henderson wrote: On 2010/08/19 11:55, Marc Espie wrote: I want to tweak the directory structure for ports. I'd like to go for a lot of stuff to ports/infrastructure/bin ports/infrastructure/lib ports/infrastructure/man Yes please. Two things; - does mk move? no. - can we add ports/infrastructure/man to man.conf? globally ? no idea. on my ports boxes: definitely.
Re: RFC: changes to ports infrastructure
On Thu, Aug 19, 2010 at 02:46:44PM +0200, Joachim Schipper wrote: On Thu, Aug 19, 2010 at 11:55:22AM +0200, Marc Espie wrote: I want to tweak the directory structure for ports. I'd like to go for a lot of stuff to ports/infrastructure/bin ports/infrastructure/lib ports/infrastructure/man the current setup (build, fetch, install, package) is my mistake, and frankly it sucks. Some of the scripts are callable outside of bsd.port.mk, and you have to remember the path. There is no standard documentation out of the scripts, and there are a few perl modules that could use one single place. I would keep db, plist, templates. I don't think they're that confusing. The only drawback is that we're going to lose cvs history. doesn't seem like such a big problem to me. Also, third party may lose compatibility as scripts move around. That's easy to solve with symlinks, so I don't care too much. Did I miss anything ? Something you may also want to consider: currently, you can't really install a partial ports tree. If all modules were kept in e.g. ports/infrastructure/modules, it may be possible to install only a hypothetical ports-infrastructure.tar.gz (like sys.tar.gz, but for ports) and a couple of individual ports (e.g. under mystuff/). This would make it very easy to upgrade just one package on a -stable system (e.g. to fix a security issue or get an updated port from -current). It would be even better if FETCH_PACKAGES could be made to work in this scenario, but that's not required. This is in no way a must-have feature, but if you're moving stuff anyway... No-go. You also need proper depends to compute default pkgnames for the package, so you mostly need the full ports tree anyways. And modules was specifically designed to be modular, with no central place... I *could* have bsd.port.mk auto-checkout directories when needed with a specific cvs script, But considering the small size of the ports tree, it is probably one more useless feature. though.
Re: RFC: changes to ports infrastructure
Okay, I've started moving a lot of things. I've moved aggressively stuff that is internal to the ports tree (all the scripts used by bsd.port.mk) and dpb3 (renamed to dpb, as I hope to finally deprecate the other dpb). I'm probably going to document this part, and then do a second pass for the rest...
new perl
if you run into fishy perl scripts, newer perl includes (as standard) autodie. a simple use autodie; and at least, stupid stuff that NEVER CHECKS error returns from open() and the like will die, die, die instead of struggling onwards...
requests for perl XS code
I could use bundled code to do the following: 1/ direct interface to mkstemp so that we don't have to use the atrocious File::Temp big monstrosity. 2/ file descriptor passing over sockets means redoing the stuff described in socket/recv/CMSG_DATA so that we could have a simple interface to grab stuff in perl. looking for prior art/someone with the time to do it as XS... 2/ would be useful for libtool: on slow machines, libtool startup time is large, so it would be beneficial to start a libtool server at start of build, and communicate with it using a (very small) c client. Passing stdout/stderr/cwd to the server over the socket seems like the most straightforward way to do stuff, especially since we wouldn't have to deal with any kind of non-transparent buffering then...
Re: requests for perl XS code
On Sat, Oct 16, 2010 at 03:54:50PM -0700, Philip Guenther wrote: On 10/16/10, Marc Espie es...@nerim.net wrote: I could use bundled code to do the following: 1/ direct interface to mkstemp so that we don't have to use the atrocious File::Temp big monstrosity. I've sent Marc a tiny mkdtemp/mkstemp/mkstemps-via-XS module, so unless he barfs all over it this one is done. 2/ file descriptor passing over sockets means redoing the stuff described in socket/recv/CMSG_DATA so that we could have a simple interface to grab stuff in perl. Marc, do you have an API in mind that someone working on this should aim for? Mostly some XS wrapper around the recvmsg/CMSG_DATA part. Then some perl glue that would allow one to grab those data, and gets the fds from it. It should probably only extract the msgs, let normal stuff grab non out-of-band data IO, and create an array of those, with level/type/data. If you want to do the send part as well, it should be reversed. What I'm talking about is mostly wrapping around recvmsg/sendmsg and CMSG_DATA pack/unpack type routines.
groff going away at some point...
... and the ports tree is actually ready to switch. Any port that requires groff just need to set USE_GROFF=Yes to - have a build depends on the groff port - have pkg_create format pages using the groff port behind the scenes. Note that this is transparent revision-wise, there's absolutely no need to bump anything, since that's a build-depends, and register-plist is already aware of the equivalence between manN/foo.N and catN/foo.0 So, it's just a question of a massive commit to tag all ports that have manpages with USE_GROFF, do a build and check that everything is fine. Then we can slowly switch everything that's happy with it back to mandoc.
Re: next step in *DEPENDS simplification
On Tue, Nov 16, 2010 at 12:29:08AM +0100, Alexander Hall wrote: On 11/15/10 19:31, Marc Espie wrote: So now we have *DEPENDS = pkgspec:pkgpath in an uniform way, with pkgspec: completely omitted when it's the default stem-* The idea is to remove as much noise as possible and it's already working great Compare: -RUN_DEPENDS= :desktop-file-utils-*:devel/desktop-file-utils \ - :shared-mime-info-=0.21:misc/shared-mime-info +RUN_DEPENDS= devel/desktop-file-utils \ + shared-mime-info-=0.21:misc/shared-mime-info but it might still be a bit too complex, especially since the most frequent spec is this stem past that version, so STEM- +RUN_DEPENDS= devel/desktop-file-utils \ + STEM-=0.21:misc/shared-mime-info still a bit of line noise. Dropping STEM- altogether looks unnatural +RUN_DEPENDS= devel/desktop-file-utils \ + =0.21:misc/shared-mime-info let's tack the number at the end instead: +RUN_DEPENDS= devel/desktop-file-utils \ + misc/shared-mime-info=0.21 So you drop the dash after the stem path here? While I do find that notation at least as readable, if not more so, it could possibly cause confusion vs the shared-mime-info-=0.21:... notation. There is no stem in there. A pkgpath is something different. For a pkgspec, the - does makes some sense, because we go from screen-3.0.2 to screen-=3.0.2 but in misc/screen there's never been any -, so misc/screen=3.02 should be okay. That said, it's likely that more verifications for valid specs might save some people some head-scratching...
DEPENDS cleanup rampage finished
I've finally finished cleaning the DEPENDS lines to a state I like. A new feature was introduced, but it's rather esoteric, and VERY special case, so I'd advise against using it for very good reasons, it's a new ports variable called PKGSPEC. Here is what happened before PKGSPEC: When you write a dependency line such as: BUILD_DEPENDS = devel/bison there is just a directory name, no pkgspec part, so the pkgspec part is automatically generated. Up until a week ago, it was ALWAYS stem-*, e.g., the above was equivalent to BUILD_DEPENDS = bison-*:devel/bison This is almost always appropriate, and there is very little case for changing it for almost any port. HOWEVER, a few ports can coexist as various versions. For instance, you can install autoconf-2.13 alongside autoconf-2.59. So, what do you do when you need a given autoconf version ? Turns out the following would be incorrect: BUILD_DEPENDS = devel/autoconf/2.59 if no autoconf is installed, then this would install autoconf-2.59. However, this is really BUILD_DEPENDS = autoconf-*:devel/autoconf/2.59 any autoconf would do. So if autoconf-2.13 was already there, then the port would happily continue building, and usually fail in atrocious ways. - PKGSPEC is something I can now put in the autoconf/2.59 port proper, I would write PKGSPEC = autoconf-2.59 and suddenly, my BUILD_DEPENDS becomes correct: BUILD_DEPENDS = devel/autoconf/2.59 will be enough to only match autoconf-2.59. This is very powerful: I can now tweak all default dependencies on a port by just adding a line to that port. This is also quite dangerous and very sneaky: when you see a DEPEND like the following, you have no obvious way of knowing whether it's autoconf-* or autoconf-2.59. (okay, if you build it, you'll find out soon enough). So it should be used very seldom. For the few cases that require it, though, it simplifies things a great deal. I did fix things for: devel/autoconf/* devel/automake/* devel/libsigc++ databases/db/* databases/db/* security/gnupg net/libdnet/* besides simplifying things, this did catch at least ten dependency bugs, where a port would not be specific enough in what it wanted. I see java, tomcat, tcl, tk, python as other potential candidates for PKGSPEC, but I don't know that software well enough to decide, maintainers will have to think about it. (I know that jeremy already decided to use it for ruby). Maybe notification-daemon-* ? I don't know. But most ports out there shouldn't need it. How do the new dependency features interact with pkgnames bumps and revisions ? Well, actually, they don't... everything that matters for register-plist is the final product, where all dependencies are rewritten as @depends and @wantlib lines. So if you end up with a pkgspec that says db-=4,5 it doesn't matter if it comes from a default or an explicit pkgspec in the dependency. HOWEVER, if you *do* change a PKGSPEC on an existing port, *every* depending port may pull the new default spec, and thus require a bump. So you need to check things very carefully. Like I said, PKGSPEC is not a simple feature, nor a feature that should be used in most cases.
Re: Allegations regarding OpenBSD IPSEC
I'm not going to comment on the mail itself, but I've seen a lot of incredibly dubious articles on the net over the last few days. - use your brains, people. Just because a guy does say so doesn't mean there's a backdoor. Ever heard about FUD ? - of course OpenBSD is going to check. Geeez!! what do you think ? - why would OpenBSD be in trouble ? where do you think *all the other IPsec implementations* come from ? (hint: 10 years ago, what was the USofA view on cryptography exports ? where is OpenBSD based. Second hint: Canada != UsOfA). - why would the FBI only target OpenBSD ? if there's a backhole in OpenBSD, which hosts some of the most paranoid Opensource developers alive, what do you think is the likelyhood similar backholes exist in, say, Windows, or MacOs, or Linux (check where their darn IPsec code comes from, damn it!) I know that a lot of the guys reading tech@ are intelligent enough to *know* all the rather obvious things I'm stating here, but it's looking like a lot of stupid, stupid web sites are using this as their *only* source of information, and do not engage their brain): if you read french, go check http://www.macgeneration.com/news/voir/180982/un-systeme-espion-du-fbi-dans-openbsd and be amazed at how clueless those writers are. Just on the off chance that those idiots will read this, and realize how stupid their generalizations are. Theo was careful enough to state facts, and I'm a huge fan of what he's done (he's decided to go fully open with this, which was a tough decision). I don't see why this would impact OpenBSD negatively without affecting any other OS... especially until we actually get proof...
Re: Allegations regarding OpenBSD IPSEC
On Fri, Dec 17, 2010 at 08:59:13AM -0700, Theo de Raadt wrote: This is not to point finger at Theo for creating all this commotion, of course; this commotion can, however, be, an unintended accident, but the fact that it came from Theo gave it a lot of credibility. Whoa, wait a second here. If you think I gave it credibility, you need to go back and read my words again. I called it an allegation, and I stick with that. I was extremely careful with my words, and you are wrong to interpret them as you do. Theo, it's hopeless. Kids these days. Can't read, can't code. If you write anything, you can be certain they will take it out of context. They don't understand what a context is. Heck, they will use the excuse that they're not native speakers to say they misunderstood. I mean, why should they make the effort ? it's so easier to take a rumor out of context, not verify the source, not verify what it says and run with it. There's NEVER an excuse for mediocrity. I'm not a native speaker, Theo isn't either. That's not a good reason for not understanding/not writing english. That's the same with code, just because you learnt to program with a bad crowd is no excuse for most of the linux and java code out there. ;-)
Re: http gzip support for ftp
On Sun, Jan 09, 2011 at 04:21:51PM -0500, Ted Unangst wrote: Downloading things can go a lot faster if the server and client support http compression. This is easily added to the ftp program's http support. It consists of two parts. Support for deflating the data we receive and support for the chunked transfer the server will use to send data to us. The chunked supported is probably useful in its own right as well. How does it work if you use http to transfer already compressed files ? Does it need special intelligence to avoid compressing things again ? Remember that a lot of people do pkg_add over http. We definitely do NOT want to recompress packages that are already .tgz...
Re: making struct __sFILE editable
On Tue, Jun 09, 2009 at 07:12:24PM +0200, Joerg Sonnenberger wrote: On Tue, Jun 09, 2009 at 10:37:29AM -0600, Theo de Raadt wrote: The fact that you had to modify standards compliant code already in the tree to make it work indicates a rather major problem. The changes for stdout / stderr / stdin at least are valid, the standard doesn't guarantee that they can be used that way. ISO C explicitly defines them as extern FILE *stderr etc. It is possible to mostly hide the FILE internals and the breakage is not as big as portrait in that thread. It is still quite painful as a lot of programs want to look at the inner workings though. There are a number of open issues with the public API for certain operations mandated by SUS, e.g. the combination of file pointer position for buffered files when sharing the file between processes etc. Joerg Well, I remember just doing some stupid extension for locale support, and spending a *lot* of time getting things to work afterwards... and this definitely was less invasive than hiding FILE internals The devil lies in the BOATLOAD of details...
Re: Thank you for making p2k9 possible!
On Sun, Oct 11, 2009 at 04:56:00PM +0200, Robert Nagy wrote: Hello p2k9 (the ports hackathon in Budapest) is on since Friday. People are working on different things like GNOME, GCC4, BluRay support or even ACPI. I would like to thank everyone who donated money to the project because the individual donors made it possible to organize this event. So ... BIG THANKS GOES TO OUR USERS, to people supporting the project even at these times. I'd also like to thank NIIF and Sun Microsystems Hungary for lending us a nice hackroom and hardware for the hackathon. Yeah, likewise. All of us currently trying to figuring out new cool things to add to OpenBSD, here in Budapest, we thank you. ;-)
Re: m4 diff, add support for -P
On Wed, Oct 14, 2009 at 05:33:28PM +0100, Stuart Henderson wrote: On 2009/10/14 08:38, Matthew Dempsky wrote: Doesn't this leak memory in the -P case? What about moving the xstrdup() call into only the !prefix_builtins path? Then you can also eliminate the const cast: Yes, Tobias Ulmer noticed this too, I didn't think of a way around it before but this makes a lot more sense - thanks. New diff incorporates your fix and jmc's feedback, and adds a simple regression test. (The existing tests still pass). Looks good. okay.
Re: Thank you for making p2k9 possible!
On Fri, Oct 16, 2009 at 10:37:11PM -0400, STeve Andre' wrote: You can see whats been happening if you are subscribed to the cvs src changes list. Offhand at least 30 new ports were added, more than 250 were updated, lots were tweaked, and the pkg_add code was worked on. Most of the tweaks done to pkg_add don't have any user-visible effects... yet. I hope to continue a wee little bit, and then you'll notice fireworks ;-)
recent pkg_add changes
There have been some big changes in pkg_add again.
Finally, there is now a 'quirks package that will be used to handle weird
update scenarios. It amounts to a Quirks.pm module that can build a quirks
object that conforms to pkg_add's interface. This can change independently:
pkg_add builds the quirks object with an API version number: either quirks
knows that API, and it can build the object, or it doesn't and it won't
build the object. The rest of pkg_add doesn't care whether there's a quirks
object or not (all changes are optional).
Note that this is not permission to do *anything* with package names. Quirks
is intended for exceptional situations. It does not happen all that often that
package change names, or that stuff becomes part of the base system...
So far, the quirks object is version 1, and it can do 3 things:
- tweak the list of updatesets to install at the start (we don't use this
for now)
- know whether a package is now part of the base system: this is handled
through a hash of the form:
stem = file:
we're dealing with removing packages named stem-*, and we will remove them
if file is installed on the system.
e.g., I have:
my $base_exceptions = {
'p5-version' = /usr/libdata/perl5/version.pm
};
for tests.
- know whether a package changed name. Again, this is a hash of the form:
stem = extra_stem
I tested this worked by updating a foo-0.0 to a bar-0.0 after adding
foo = bar
Please get any special changes you want through me, for the time being.
Mid-term, the not really code of Quirks is probably going to move to a
separate list. For now, I'm quite happy adding entries to those hashes...
Re: Automatic package mirror discovery implementation for pkg_add(1) tool
On Sat, Dec 05, 2009 at 10:47:25PM +0300, Igor Zinovik wrote:
if (-e /var/db/pkg/ftpmirror.cache) {
open my $fh, '', /var/db/pkg/ftpmirror.cache or
die(Permission denied);
@mirrors = $fh;
close $fh;
print $mirrors[0];
exit;
}
...
open $fh, '', /var/db/pkg/ftpmirror.cache or
die(Cannot create mirror cache file);
you can't do this. /var/db/pkg only holds packages. This will confuse
pkg_add when you try to update things.
Apart from this, this is a really tough problem, because of infrastructure
issues. Basically, our mirrors are not that reliable, and the closest
one often won't have the packages you need... which is a reason why it's
mostly some user settings...
perl and profiling
If you people ever need to profile some large perl code, I think I can recommend Devel::NYTProf, it works much faster and gives much more in-depth results than Devel::Prof. Granted, big runs of pkg_add are not exactly average, but still, Devel::Prof was generating 2G traces, and dprofpp was taking over an hour to produce its results. On the same test, Devel::NYTprof generated 200M traces, and took 15 minutes to produce very much detailed reports...
Re: xargs -0 and -L
On Fri, Feb 05, 2010 at 12:14:51PM +0200, Atte Peltomdki wrote: On Fri, Feb 05, 2010 at 02:49:16AM +0100, Ingo Schwarze wrote: Atte Peltomdki wrote on Thu, Feb 04, 2010 at 12:48:47PM +0200: On Tue, Feb 02, 2010 at 09:32:43PM +0100, Ingo Schwarze wrote: Antti Harri wrote on Tue, Feb 02, 2010 at 07:31:57PM +0200: xargs' -L switch isn't working when using -0 flag. After checking POSIX.1 (2008), Where did you find it btw? I can't find 2008 edition anywhere that wouldn't require registration and whatnot. single unix, opengroup... don't look for posix.
Re: Possible issue with srand or rand in base?
On Sun, Feb 07, 2010 at 01:59:33PM -0500, Brad Tilley wrote: I wrote a small cpp application to generate randomish passwords. It compiles and runs OK on OpenBSD, however, it does not seem to create random strings (the first and last chars seldom ever change, etc). The same code compiles and runs on Linux and Windows and *does* produce randomish strings (no often repeating chars). The source code is small and is contained in a single file. I placed it here along with binaries for OpenBSD and Windows: http://16systems.com/downloads One thing which is *really* annoying is that you can't even leave your source around for a day or two for other people to at least see it and join in the discussion. This is just another bad point for you. Definitely no cookie.
Re: Possible issue with srand or rand in base?
On Mon, Feb 08, 2010 at 08:10:11AM -0500, Brad Tilley wrote: I placed the GUI version there are source.cpp. I don't have the simpler, non-GUI version that I posted yesterday, but the use of srand and rand are the same in both examples. The GUI version compiles on OpenBSD if you have fltk installed from ports. I only wrote the simpler version to demonstrate the difference I was seeing. Wow, that's fucked. Learn how to use vector, your use of map is incredibly inefficient. And even about rand(), any text book about random number generators will tell you that it's a bad one, and that you don't ever use it with modulo, since the low-level bits are even worse than the high-level bits.
Re: ports.7
On Wed, Mar 03, 2010 at 04:46:18PM -0800, J.C. Roberts wrote:
Boolean variables should be set to Yes instead of simply being
defined, for uniformity and future compatibility.
This takes time to finish changing.
1.) The majority of the scripts in ports/infrastructure do not
bother to check /etc/mk.conf at all, so saying or and also seems
unwise without context. The best answer seems to be to preface the
statement with, When running make ...
This ought to change. It's just a bit more expensive to do it in scripts,
as it requires running make.
mk.conf is *NOT* a config file, it is a piece of Makefile !
5.) Considering the precedence set by other /etc/*.conf files like
rc.conf, using all caps of YES or NO for true/false would be more
consistent. --Since Marc is the main chef of the ports system, this
should probably be his call. Chefs' choice and all that.
No, I hate YES. I explicitly changed it because I hate it.
You can use YES if you want for your own stuff, but make has
${V:L} explicitly to allow for any kindofase.
6.) Though it mentions the private _* variables, it does not list
them, so this results in basically outlawing all variables which start
with and underscore. --thoughts?
Duh, this is the idea, yes.
You'll notice that the libc documentation does not list the variables
starting with __, and that the C standard only say that those are off-limits.
---
Index: ports.7
===
RCS file: /cvs/src/share/man/man7/ports.7,v
retrieving revision 1.79
diff -u ports.7
--- ports.7 11 Oct 2009 21:07:06 - 1.79
+++ ports.7 4 Mar 2010 00:27:56 -
@@ -242,7 +242,7 @@
Defining
.Ev NO_CHECKSUM
to
-.Dv Yes
+.Qq YES
No, diff is definitely not okay.
Re: ports.7
On Thu, Mar 04, 2010 at 06:14:52AM -0800, J.C. Roberts wrote: On Thu, 4 Mar 2010 09:38:12 +0100 Marc Espie es...@nerim.net wrote: On Wed, Mar 03, 2010 at 04:46:18PM -0800, J.C. Roberts wrote: Boolean variables should be set to Yes instead of simply being defined, for uniformity and future compatibility. This takes time to finish changing. 1.) The majority of the scripts in ports/infrastructure do not bother to check /etc/mk.conf at all, so saying or and also seems unwise without context. The best answer seems to be to preface the statement with, When running make ... This ought to change. It's just a bit more expensive to do it in scripts, as it requires running make. The first hurdle for me is checking to make sure all the scripts use the environment variables. Once those patches are in, we can revisit the /etc/mk.conf, make, and performance issues. A lot of those scripts are old stuff, which is probably going to vanish soon, since the newer infrastructure handles things in a consistent way. mk.conf is *NOT* a config file, it is a piece of Makefile ! Yep, it's a Makefile typically used to store local configuration. ;) And the only way to parse it correctly is... to run make. Which can be a bit expensive. But this point has been noted, and I intend to see whether I can make things a bit more consistent after 4.7 is out the door. I had a funny feeling about the capitalization. It seemed someone had obviously used Yes by intent in spite of the precedence for using YES --I left the capitalization as you prefer, but made sure everything is quoted identically with straight double quotes. I actually don't see the points of double quotes. Why double quotes ? In makefiles, or shell script, you often do not need any quotes at all. The .Sq macro was chosen on purpose, because it does not match actual syntax, and forces people to think about what they're about to do.
Re: git vs. $OpenBSD$
On Sun, Apr 25, 2010 at 03:01:31PM +0200, Marc Espie wrote: I think I finally figured out how to work with git for small subtrees, and not get annoyed with the $OpenBSD$ keyword expansion... - define a filter in your git attributes, that's either $GITDIR/info/attributes or ~/.gitattributes: * filter=openbsd PS: apparently, ~/.gitattributes does not exist. This part has to live inside the git repo you want to use...
Re: git vs. $OpenBSD$
On Sun, Apr 25, 2010 at 03:01:31PM +0200, Marc Espie wrote: - create a script in your PATH that cleans up things, in my case, ~/bin/openbsd-zap #! /usr/bin/perl -p # what ? you expected another scripting language s/\$OpenBSD\:.*?\$/\$OpenBSD\$/g; s/[ \t]+$//; # jmc's gonna love that PS2: the following script handles more situations: #! /usr/bin/perl -p s/\$(OpenBSD|Mdocdate|Id)\:.*?\$/\$$1\$/gs; s/[ \t]+$//;
Re: [PATCH] update zlib to 1.2.8
Updating zlib correctly is surprisingly difficult. It's not just a question of whipping out the new version and doing a diff. You should make tests. Like, you missed the #ifdef SMALL at first. Those tests include making a full release, and making sure it still works. It's also likely some of those local diffs are there for some specific architectures. If you don't have them, you have to ask for tests. I'm not saying it's not doable. But the fact you don't see the newer zlib in our tree kind-of indicates it takes quite a bit more testing than you would think.
Re: man, man.conf and /usr/ports/infrastructure/man
The rules for locking base and ports are not the same. Having the build infrastructure in-ports means it can be fixed at the same time as ports. Having it in-base means yet another possibility for synchronization issues between base and ports. Likewise for putting the man pages inside a package. I don't say it's not possible, but the man pages will stay inside the ports tree, we won't roll yet another distfiles for those. So, if you're out-of-synch, again, you lose. Basically, infrastructure/mk, infrastructure/bin and infrastructure/man are always in-synch. Also, requiring to install some of these is going to be a problem for dpb builds. Unnecessary out-of-synch and engineering issue. I already have ways too much to deal with to start building yet another ziggurat. That shit is completely unnecessary right now.
Re: id: add -c option
On Mon, May 18, 2015 at 03:12:24PM -0600, Todd C. Miller wrote:
This adds id -c to display a user's login class. If no user is
specified, it looks up the passwd entry based on the real uid and
displays the corresponding login class.
This is similar to id -c in FreeBSD (but they keep the login class
in the kernel).
- todd
Index: usr.bin/id/id.1
===
RCS file: /cvs/src/usr.bin/id/id.1,v
retrieving revision 1.17
diff -u -p -u -r1.17 id.1
--- usr.bin/id/id.1 3 Sep 2010 11:09:29 - 1.17
+++ usr.bin/id/id.1 18 May 2015 16:47:46 -
@@ -43,6 +43,9 @@
.Nm id
.Op Ar user
.Nm id
+.Fl c
+.Op Ar user
+.Nm id
.Fl G Op Fl n
.Op Ar user
.Nm id
@@ -70,6 +73,9 @@ In this case, the real and effective IDs
.Pp
The options are as follows:
.Bl -tag -width Ds
+.It Fl c
+Display the login class of the real user ID or the specified
+.Ar user .
.It Fl G
Display the different group IDs (effective, real and supplementary)
as whitespace separated numbers, in no particular order.
Index: usr.bin/id/id.c
===
RCS file: /cvs/src/usr.bin/id/id.c,v
retrieving revision 1.22
diff -u -p -u -r1.22 id.c
--- usr.bin/id/id.c 16 Jan 2015 06:40:08 - 1.22
+++ usr.bin/id/id.c 18 May 2015 21:10:13 -
@@ -38,6 +38,7 @@
#include string.h
#include unistd.h
#include limits.h
+#include login_cap.h
void current(void);
void pretty(struct passwd *);
@@ -52,12 +53,12 @@ main(int argc, char *argv[])
{
struct group *gr;
struct passwd *pw;
- int Gflag, ch, gflag, nflag, pflag, rflag, uflag;
+ int ch, cflag, Gflag, gflag, nflag, pflag, rflag, uflag;
uid_t uid;
gid_t gid;
const char *opts;
- Gflag = gflag = nflag = pflag = rflag = uflag = 0;
+ cflag = Gflag = gflag = nflag = pflag = rflag = uflag = 0;
if (strcmp(getprogname(), groups) == 0) {
Gflag = 1;
@@ -72,10 +73,13 @@ main(int argc, char *argv[])
if (argc 1)
usage();
} else
- opts = Ggnpru;
+ opts = cGgnpru;
while ((ch = getopt(argc, argv, opts)) != -1)
switch(ch) {
+ case 'c':
+ cflag = 1;
+ break;
case 'G':
Gflag = 1;
break;
@@ -101,7 +105,7 @@ main(int argc, char *argv[])
argc -= optind;
argv += optind;
- switch (Gflag + gflag + pflag + uflag) {
+ switch (cflag + Gflag + gflag + pflag + uflag) {
case 1:
break;
case 0:
@@ -117,6 +121,16 @@ main(int argc, char *argv[])
pw = *argv ? who(*argv) : NULL;
+ if (cflag) {
+ if (pw == NULL)
+ pw = getpwuid(getuid());
+ if (pw != NULL pw-pw_class != NULL *pw-pw_class != '\0')
+ (void)printf(%s\n, pw-pw_class);
+ else
+ (void)printf(%s\n, LOGIN_DEFCLASS);
+ exit(0);
+ }
+
if (gflag) {
gid = pw ? pw-pw_gid : rflag ? getgid() : getegid();
if (nflag (gr = getgrgid(gid)))
@@ -325,6 +339,7 @@ usage(void)
(void)fprintf(stderr, usage: whoami\n);
} else {
(void)fprintf(stderr, usage: id [user]\n);
+ (void)fprintf(stderr,id -c [user]\n);
(void)fprintf(stderr,id -G [-n] [user]\n);
(void)fprintf(stderr,id -g [-nr] [user]\n);
(void)fprintf(stderr,id -p [user]\n);
Since I asked for it, I obviously approve of this change.
Re: chroot: add -c to use login class with -u
On Mon, May 18, 2015 at 04:30:33PM -0600, Todd C. Miller wrote:
Currently, chroot -u doesn't use the settings in /etc/login.conf.
This adds a -c option to apply the via setusercontext(). We can't
use LOGIN_SETALL since the uid change has to happen after chroot(2)
and the groups may be specified via the -g option.
Open questions:
1) Should this just be default behavior with -u? Are there cases
when you would *not* want to set the priority and resouce limits
based on the target user when one is specified?
Color me surprised, I just looked at posix, and discovered that chroot is
*not* a standard utility. So yeah, that would even make sense to me.
2) Does it make sense to apply LOGIN_SETPATH, LOGIN_SETPATH
and LOGIN_SETUMASK or are we better off with just LOGIN_SETPRIORITY
and LOGIN_SETRESOURCES? Ultimately it depends on whether you
expect the chroot'd environment to be setup like a login session
or not. We don't invoke the shell as a login shell though.
For my use case, everything can be set up as a login session... Note that
I would use the chroot -c -u user /directory cmd directly, so there is
no login shell involved.
Opinions?
Looks fine to me as it currently stands.
- todd
Index: usr.sbin/chroot/chroot.8
===
RCS file: /cvs/src/usr.sbin/chroot/chroot.8,v
retrieving revision 1.14
diff -u -p -u -r1.14 chroot.8
--- usr.sbin/chroot/chroot.8 8 Jul 2010 06:52:30 - 1.14
+++ usr.sbin/chroot/chroot.8 18 May 2015 20:02:55 -
@@ -37,6 +37,7 @@
.Nd change root directory
.Sh SYNOPSIS
.Nm chroot
+.Op Fl c
.Op Fl g Ar group,group,...
.Op Fl u Ar user
.Ar newroot
@@ -56,6 +57,13 @@ command is restricted to the superuser.
.Pp
The options are as follows:
.Bl -tag -width Ds
+.It Fl c
+Apply the user's login class as defined in
+.Xr login.conf 5
+before executing
+.Ar command .
+This option may only be used in conjunction with
+.Fl u .
.It Fl g Ar group,group,...
Override the primary and supplemental group IDs.
The primary group ID is set to the first group in the list.
@@ -94,6 +102,7 @@ is used.
.El
.Sh SEE ALSO
.Xr ldd 1 ,
+.Xr login.conf 5 ,
.Xr group 5 ,
.Xr passwd 5 ,
.Xr environ 7
Index: usr.sbin/chroot/chroot.c
===
RCS file: /cvs/src/usr.sbin/chroot/chroot.c,v
retrieving revision 1.13
diff -u -p -u -r1.13 chroot.c
--- usr.sbin/chroot/chroot.c 27 Oct 2009 23:59:51 - 1.13
+++ usr.sbin/chroot/chroot.c 18 May 2015 19:56:15 -
@@ -35,8 +35,10 @@
#include errno.h
#include grp.h
#include limits.h
+#include login_cap.h
#include paths.h
#include pwd.h
+#include stdbool.h
#include stdio.h
#include stdlib.h
#include string.h
@@ -50,16 +52,21 @@ main(int argc, char **argv)
{
struct group*grp;
struct passwd *pwd;
+ login_cap_t *lc = NULL;
const char *shell;
char*user, *group, *grouplist;
gid_t gidlist[NGROUPS_MAX];
int ch, ngids;
+ boolcflag = false;
ngids = 0;
pwd = NULL;
user = grouplist = NULL;
- while ((ch = getopt(argc, argv, g:u:)) != -1) {
+ while ((ch = getopt(argc, argv, cg:u:)) != -1) {
switch(ch) {
+ case 'c':
+ cflag = true;
+ break;
case 'u':
user = optarg;
if (*user == '\0')
@@ -83,6 +90,12 @@ main(int argc, char **argv)
if (user != NULL (pwd = getpwnam(user)) == NULL)
errx(1, no such user `%s', user);
+ if (cflag) {
+ if (pwd == NULL)
+ errx(1, login class requires a user);
+ lc = login_getclass(pwd-pw_class);
+ }
+
while ((group = strsep(grouplist, ,)) != NULL) {
if (*group == '\0')
continue;
@@ -104,6 +117,11 @@ main(int argc, char **argv)
err(1, setgid);
if (initgroups(user, pwd-pw_gid) == -1)
err(1, initgroups);
+ if (lc != NULL) {
+ const int flags = LOGIN_SETALL
~(LOGIN_SETGROUP|LOGIN_SETLOGIN|LOGIN_SETUSER);
+ if (setusercontext(lc, pwd, pwd-pw_uid, flags) == -1)
+ err(1, setusercontext);
+ }
}
if (chroot(argv[0]) != 0 || chdir(/) != 0)
pkg_* new functionality: arch, version expansion
was requested by sthen@
works with pkg.conf and PKG_PATH, similar sequences as for dpb:
%a - architecture
%c - either snapshots or version number
%v - always version number.
please test/comment
just tried it after rewriting:
installpath = http://ftp.fr.openbsd.org/pub/OpenBSD/%c/packages/%a/
in pkg.conf
Index: OpenBSD/PackingElement.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackingElement.pm,v
retrieving revision 1.239
diff -u -p -r1.239 PackingElement.pm
--- OpenBSD/PackingElement.pm 25 Feb 2015 16:37:15 - 1.239
+++ OpenBSD/PackingElement.pm 15 Apr 2015 15:53:09 -
@@ -1750,6 +1750,15 @@ sub stringize($)
my ($machine_arch, $arch);
+sub arch
+{
+ if (!defined $arch) {
+ my $cmd = OpenBSD::Paths-uname. -m;
+ chomp($arch = `$cmd`);
+ }
+ return $arch;
+}
+
sub check
{
my ($self, $forced_arch) = @_;
@@ -1768,11 +1777,7 @@ sub check
chomp($machine_arch = `$cmd`);
}
return 1 if $ok eq $machine_arch;
- if (!defined $arch) {
- my $cmd = OpenBSD::Paths-uname. -m;
- chomp($arch = `$cmd`);
- }
- return 1 if $ok eq $arch;
+ return 1 if $ok eq arch();
}
return;
}
Index: OpenBSD/PackageRepository/Installed.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackageRepository/Installed.pm,v
retrieving revision 1.29
diff -u -p -r1.29 Installed.pm
--- OpenBSD/PackageRepository/Installed.pm 15 Jun 2014 23:49:51 -
1.29
+++ OpenBSD/PackageRepository/Installed.pm 15 Apr 2015 15:53:09 -
@@ -26,6 +26,40 @@ use warnings;
package OpenBSD::PackageRepositoryBase;
+my ($version, $current);
+
+sub expand_locations
+{
+ my ($class, $string, $state) = @_;
+ if ($string eq '%a') {
+ require OpenBSD::PackingElement;
+ return OpenBSD::PackingElement::Arch::arch();
+ } else {
+ if (!defined $version) {
+ require OpenBSD::Paths;
+ open my $cmd, '-|',
+ OpenBSD::Paths-sysctl, '-n', 'kern.version';
+ my $line = $cmd;
+ close($cmd);
+ if ($line =~ m/^OpenBSD (\d\.\d)(\S*)\s/) {
+ $version = $1;
+ if ($2 eq '-current') {
+ $current = 'snapshots';
+ } else {
+ $current = $version;
+ }
+ } else {
+ $state-fatal(Can't figure out version);
+ }
+ }
+ if ($string eq '%c') {
+ return $current;
+ } else {
+ return $version;
+ }
+ }
+}
+
sub parse_url
{
my ($class, $r, $state) = @_;
@@ -40,6 +74,7 @@ sub parse_url
$$r = '';
}
+ $path =~ s/\%[vac]/$class-expand_locations($, $state)/ge;
$path .= '/' unless $path =~ m/\/$/;
bless { path = $path, state = $state }, $class;
}
Re: pkg_* new functionality: arch, version expansion
On Wed, Apr 15, 2015 at 08:39:15PM +0200, Sebastian Reitenbach wrote: I've seen at least -beta versions short before release locks. Not sure if there are -alpha too? Those should be treated same way as snapshots, I think. This is the exact same logic that's in fw_update, actually... so if one is wrong, the other one would be too ?
Re: man, man.conf and /usr/ports/infrastructure/man
On Sun, Apr 12, 2015 at 04:43:31PM -0400, dan mclaughlin wrote: it seems that /usr/ports/infrastructure/man is not searched by default, and there is no example in man.conf for it. given that some pages like dpb(1) are there, and referenced thruout a number of pages (eg bsd.port.mk), this seems an oversight. on a releated note, it also seems that ports/infrastructure/man/mandoc.db is also not created by default. Yep, it was removed as an executive decision by Theo. Among numerous reasons, the main reason being that it's not part of the base system, and possibly subject to containing bullshit that would play havoc with your terminal or fuck up the makewhatis process.
Re: Tying time_t to pacman
On Mon, Apr 13, 2015 at 06:10:36PM -0600, Theo de Raadt wrote: I don't normally do this. http://motherboard.vice.com/read/32-bit-integers-and-why-old-computers-matter This does a really good job educating people on integer overflow :) In our non-pacman universe, the ports guys really did a good job mopping up the 32bit time_t issues. But we live in a pacman universe! and level 256 was simply awesome (Or amazing! -- wait, that's henning, not pacman). I contend we need more bugs like that, I intend to reintroduce integer overflow bugs in the nethack level generator, now that we had to get rid of the misaligned time_t *. Don't you think a sokoban level where you don't see half the level would be legendary ?
Re: jail_bin_add: script to add binary and libs to chroot
On Mon, Jun 08, 2015 at 03:37:27PM +0200, Landry Breuil wrote: On Mon, Jun 08, 2015 at 02:59:28PM +0200, Marc Espie wrote: On Mon, Jun 08, 2015 at 01:46:17AM -0400, dan mclaughlin wrote: i figure this should be useful to some. any nits welcome. Unfortunately, this will become increasingly useless in gtk-land. Compare ldd firefox vs a ktrace of the running binary... :( Totally pointless example - firefox is a very specific case, in that case you want ldd /usr/local/lib/firefox-*/libxul.so.* i dont know of many gtk applications doing this (bundling everything in a dlopen'ed library with tons of linking hacks to improve startup times...) I took firefox because it is the most blatant example, but Antoine told me about it in general before I took a look at firefox.
Re: jail_bin_add: script to add binary and libs to chroot
On Mon, Jun 08, 2015 at 01:46:17AM -0400, dan mclaughlin wrote: i figure this should be useful to some. any nits welcome. Unfortunately, this will become increasingly useless in gtk-land. Compare ldd firefox vs a ktrace of the running binary... :(
Re: ftp(1) rewrite
On Mon, Jun 01, 2015 at 05:25:15PM -0400, Ted Unangst wrote: Stuart Henderson wrote: On 2015/06/01 10:20, patrick keshishian wrote: On 6/1/15, Sunil Nimmagadda su...@nimmagadda.net wrote: On Thu, May 21, 2015 at 11:16:09PM -0400, Ted Unangst wrote: screw ftp. just make a new util http, that just does http. Sorry, it's not good enough to replace ftp(1) for system use without ftp. Like it or not, ports fetches need FTP and can't really rely on installing something for ports to do that. It's not obvious to me why this would be the case. The ports tree already depends on various external programs to extract xz, cab, etc. We don't have any infrastructure for FETCH_DEPENDS, and I'm very loathe to make things even MORE complex than they already are...
chromium does not handle some video streams that firefox does
For instance, https://www.youtube.com/watch?v=J_8mdH20qTQ Youtube-dl makes the reason pretty obvious: https://www.youtube.com/watch?v=J_8mdH20qTQ [youtube] J_8mdH20qTQ: Downloading webpage [youtube] J_8mdH20qTQ: Extracting video information [youtube] J_8mdH20qTQ: Downloading DASH manifest [download] Destination: Remove cat before flight-J_8mdH20qTQ.f137.mp4 [download] 100% of 58.62MiB in 01:29 [download] Destination: Remove cat before flight-J_8mdH20qTQ.f140.m4a [download] 100% of 2.68MiB in 00:04 [ffmpeg] Merging formats into Remove cat before flight-J_8mdH20qTQ.mp4 Deleting original file Remove cat before flight-J_8mdH20qTQ.f137.mp4 (pass -k to keep) Deleting original file Remove cat before flight-J_8mdH20qTQ.f140.m4a (pass -k to keep) (e.g., split video/audio) There is definitely something different going on, because firefox can stream this, and chrome (on windows) has no issue as well...
Re: pkg_info: print used repos
On Tue, Jun 23, 2015 at 08:53:15PM -0600, Theo de Raadt wrote:
- $state-handle_options('cCdfF:hIKLmPQ:qr:RsSUe:E:Ml:aAt',
+ $state-handle_options('cCdfF:hIKLmpPQ:qr:RsSUe:E:Ml:aAt',
Starting to look a lot like ls.
What a coincidence. It's used to list things. So is ls.
Re: pkg_info: print used repos
On Wed, Jun 24, 2015 at 11:59:41AM +0200, ludovic coues wrote:
2015-06-24 11:40 GMT+02:00 Marc Espie es...@nerim.net:
On Tue, Jun 23, 2015 at 08:53:15PM -0600, Theo de Raadt wrote:
- $state-handle_options('cCdfF:hIKLmPQ:qr:RsSUe:E:Ml:aAt',
+ $state-handle_options('cCdfF:hIKLmpPQ:qr:RsSUe:E:Ml:aAt',
Starting to look a lot like ls.
What a coincidence. It's used to list things. So is ls.
Maybe Theo was hoping someone would trim ls of all the unused option.
Not surprisingly. But getting pkg_info cleaned up is not very simple, and
also it's definitely not my main priority now.
I would very much prefer someone to track down the sometimes issue where
pkg_add burps with /var/db/pkg out of synch. I also need to rewrite
Vstat.pm to handle some things it can. I have a race condition in dpb that
I want to fix, and my chroot script isn't finished yet.
There are many things which are not perfect in pkg_add nor dpb. Getting things
better is happening but it's a slow process.
Re: faq/current.html: Mention sudo removal
On Sun, Jul 05, 2015 at 05:52:54PM -0600, Theo de Raadt wrote: Why is sudo being removed from base? It is pretty useful. I imagine many use sudo Sorry, we are making room in the tree so that lynx can come back. An actual serious answer would make sense at this point, you know...
Re: [Patch] New item to the Migrating to OpenBSD guide
On Mon, Jun 29, 2015 at 02:41:16AM +0300, li...@wrant.com wrote: Lack of resources is about the only good reason there is for not providing binary updates. That's not true. Further, base + packages are updated frequently in snapshots, which is exactly a binary upgrade path for users without worry. This works exceedingly well and is well stated in the following current and stable pages. You're an idiot. Work on current is something that naturally occurs. It does not mean there are enough resources to *duplicate that work* and do the same on stable. I don't understand how you can be so stupid, especially since you admitted in a further email to using OpenBSD for years. Dude, it's well known and documented that there aren't enough people around to - figure out what needs to be committed to stable - build the resulting pieces (which *requires* an extra set of machines that run stable) - push them out. Basically, if you do that, you take away from the work being done on current. We're already stretched thin enough as-is.
Re: Do not hide actual error for setusercontext
On Tue, Jul 28, 2015 at 04:10:37PM +0300, Vadim Zhukov wrote: I see no point in hidding underlying error message from user. Found while preparing chroot for doas run tests. Okay? -- WBR, Vadim Zhukov Index: doas.c === RCS file: /cvs/src/usr.bin/doas/doas.c,v retrieving revision 1.27 diff -u -p -r1.27 doas.c --- doas.c26 Jul 2015 22:44:33 - 1.27 +++ doas.c28 Jul 2015 13:08:05 - @@ -417,7 +424,7 @@ main(int argc, char **argv, char **envp) if (setusercontext(NULL, pw, target, LOGIN_SETGROUP | LOGIN_SETPRIORITY | LOGIN_SETRESOURCES | LOGIN_SETUMASK | LOGIN_SETUSER) != 0) - errx(1, failed to set user context for target); + err(1, failed to set user context for target); syslog(LOG_AUTHPRIV | LOG_INFO, %s ran command as %s: %s, myname, pw-pw_name, cmdline); Okay from me, although tedu@ should have the last word.
pkg_add bugs fixed
I think I finally found the bug that was triggering on cups updates (the infamous +REQUIRED_BY not found error). I got a fix in -current (commit to Dependencies.pm), e.g., post 5.8. So if you still run into that issue on a -current system, I would really love to hear about that.
Re: doas with a timeout
On Mon, Jul 27, 2015 at 02:55:34PM +0300, Gregory Edigarov wrote: Thanks for showing that to me, Marc. Will think on how to implement that in a more secure way. I think I need to add a token based check? Correct? I don't think you get what I'm saying. I don't think there's any way to implement that such that it would be: - secure, - simple. thus, I don't think it belongs in doas. This is what sthen@ is also trying to tell you. We removed sudo from the base tree for a main reason: excessive complexity.
Re: doas with a timeout
On Mon, Jul 27, 2015 at 10:54:02AM +0300, Gregory Edigarov wrote:
Hi,
sudo was having a nice feature of not overwhelming the user with password
prompts (cookies :-) ).
This diff is adding this back to doas(1).
Index: doas.c
===
RCS file: /cvs/src/usr.bin/doas/doas.c,v
retrieving revision 1.27
diff -c -r1.27 doas.c
*** doas.c26 Jul 2015 22:44:33 -1.27
--- doas.c27 Jul 2015 06:50:32 -
***
*** 18,29
--- 18,31
#include sys/types.h
#include sys/stat.h
+ #include fcntl.h
#include limits.h
#include login_cap.h
#include bsd_auth.h
#include string.h
#include stdio.h
#include stdlib.h
+ #include time.h
#include err.h
#include unistd.h
#include pwd.h
***
*** 52,57
--- 54,82
return cnt;
}
+ int
+ checktimeout (const char *username)
+ {
+ char path[PATH_MAX];
+ struct stat stinfo;
+ time_t tv;
+ int fh;
+
+ snprintf((char *)path,PATH_MAX-1,/tmp/doas.timestamp.%s,username);
+ tv = time((time_t *)NULL);
+
+ if (!stat(path,(struct stat *)stinfo)
(tv-stinfo.st_mtim.tv_sec)pw_timeout)
+ return 1;
+ else {
+ fh = creat(path,S_IRUSR|S_IWUSR);
+ close(fh);
+ return 0;
+ }
+ return 0;
+ }
+
+
+
static int
parseuid(const char *s, uid_t *uid)
{
***
*** 399,405
failed command for %s: %s, myname, cmdline);
fail();
}
!
if (!(rule-options NOPASS)) {
if (nflag)
errx(1, Authorization required);
--- 424,430
failed command for %s: %s, myname, cmdline);
fail();
}
! if (pw_timeout !checktimeout(pw-pw_name)) {
if (!(rule-options NOPASS)) {
if (nflag)
errx(1, Authorization required);
***
*** 409,414
--- 434,440
fail();
}
}
+ }
envp = copyenv((const char **)envp, rule);
pw = getpwuid(target);
Index: doas.conf.5
===
RCS file: /cvs/src/usr.bin/doas/doas.conf.5,v
retrieving revision 1.11
diff -c -r1.11 doas.conf.5
*** doas.conf.523 Jul 2015 15:26:37 -1.11
--- doas.conf.527 Jul 2015 06:50:32 -
***
*** 22,31
.Sh DESCRIPTION
The
.Xr doas 1
! utility executes commands as other users according to the rules
in the
.Nm
configuration file.
.Pp
The rules have the following format:
.Bd -ragged -offset indent
--- 22,42
.Sh DESCRIPTION
The
.Xr doas 1
! utility executes commands as other users according to the rules and
options
in the
.Nm
configuration file.
+ .Sh OPTIONS
+ .Pp
+ The configuration file currently accepts one option:
+ .Bd -ragged -offset indent
+ .Ic timeout
+ tsec
+ .Pp
+ By default
+ .Ic doas
+ prompts for password on every execution. This option sets timeout for
password reprompt to the tsec seconds value.
+ .Sh RULES
.Pp
The rules have the following format:
.Bd -ragged -offset indent
***
*** 113,118
--- 124,132
.Bd -literal -offset indent
# Non-exhaustive list of variables needed to
# build release(8) and ports(7)
+ # timeout is optional
+ timeout 300 # sets the password reprompt time out to 5 minutes
+
permit nopass keepenv { \e
FTPMODE PKG_CACHE PKG_PATH SM_PATH SSH_AUTH_SOCK \e
DESTDIR DISTDIR FETCH_CMD FLAVOR GROUP MAKE MAKECONF \e
Index: doas.h
===
RCS file: /cvs/src/usr.bin/doas/doas.h,v
retrieving revision 1.4
diff -c -r1.4 doas.h
*** doas.h24 Jul 2015 06:36:42 -1.4
--- doas.h27 Jul 2015 06:50:32 -
***
*** 10,15
--- 10,16
const char **envlist;
};
+ extern time_t pw_timeout;
extern struct rule **rules;
extern int nrules, maxrules;
extern int parse_errors;
Index: parse.y
===
RCS file: /cvs/src/usr.bin/doas/parse.y,v
retrieving revision 1.10
diff -c -r1.10 parse.y
*** parse.y24 Jul 2015 06:36:42 -1.10
--- parse.y27 Jul 2015 06:50:32 -
***
*** 17,22
--- 17,23
%{
#include sys/types.h
+ #include sys/limits.h
#include ctype.h
#include unistd.h
#include stdint.h
***
*** 24,29
--- 25,31
#include stdio.h
#include string.h
#include err.h
+ #include errno.h
#include doas.h
***
*** 45,50
--- 47,53
FILE *yyfp;
+ time_t pw_timeout;
struct rule **rules;
int nrules, maxrules;
int parse_errors = 0;
***
*** 56,62
%}
%token TPERMIT TDENY TAS TCMD TARGS
! %token TNOPASS TKEEPENV
%token
doas -n
I don't think it falls on the side of bloat, and it's a pretty nifty option
to sudo...
Index: doas.1
===
RCS file: /build/data/openbsd/cvs/src/usr.bin/doas/doas.1,v
retrieving revision 1.10
diff -u -p -r1.10 doas.1
--- doas.1 21 Jul 2015 17:49:33 - 1.10
+++ doas.1 26 Jul 2015 11:13:52 -
@@ -21,7 +21,7 @@
.Nd execute commands as another user
.Sh SYNOPSIS
.Nm doas
-.Op Fl s
+.Op Fl ns
.Op Fl C Ar config
.Op Fl u Ar user
.Ar command
@@ -38,6 +38,10 @@ Parse and check the configuration file
.Ar config ,
then exit.
No command is executed.
+.It Fl n
+Non interactive mode, fail if
+.Nm
+would prompt for password.
.It Fl s
Execute the shell from
.Ev SHELL
Index: doas.c
===
RCS file: /build/data/openbsd/cvs/src/usr.bin/doas/doas.c,v
retrieving revision 1.21
diff -u -p -r1.21 doas.c
--- doas.c 24 Jul 2015 06:36:42 - 1.21
+++ doas.c 26 Jul 2015 11:13:52 -
@@ -295,9 +295,10 @@ main(int argc, char **argv, char **envp)
int ngroups;
int i, ch;
int sflag = 0;
+ int nflag = 0;
uid = getuid();
- while ((ch = getopt(argc, argv, C:su:)) != -1) {
+ while ((ch = getopt(argc, argv, C:nsu:)) != -1) {
switch (ch) {
case 'C':
setresuid(uid, uid, uid);
@@ -307,6 +308,9 @@ main(int argc, char **argv, char **envp)
if (parseuid(optarg, target) != 0)
errx(1, unknown user);
break;
+ case 'n':
+ nflag = 1;
+ break;
case 's':
sflag = 1;
break;
@@ -361,7 +365,7 @@ main(int argc, char **argv, char **envp)
}
if (!(rule-options NOPASS)) {
- if (!auth_userokay(myname, NULL, NULL, NULL)) {
+ if (nflag || !auth_userokay(myname, NULL, NULL, NULL)) {
syslog(LOG_AUTHPRIV | LOG_NOTICE,
failed password for %s, myname);
fail();
Re: doas -n
On Sun, Jul 26, 2015 at 02:27:55PM +0300, Vadim Zhukov wrote:
2015-07-26 14:15 GMT+03:00 Marc Espie es...@nerim.net:
I don't think it falls on the side of bloat, and it's a pretty nifty option
to sudo...
Index: doas.1
===
RCS file: /build/data/openbsd/cvs/src/usr.bin/doas/doas.1,v
retrieving revision 1.10
diff -u -p -r1.10 doas.1
--- doas.1 21 Jul 2015 17:49:33 - 1.10
+++ doas.1 26 Jul 2015 11:13:52 -
@@ -21,7 +21,7 @@
.Nd execute commands as another user
.Sh SYNOPSIS
.Nm doas
-.Op Fl s
+.Op Fl ns
.Op Fl C Ar config
.Op Fl u Ar user
.Ar command
@@ -38,6 +38,10 @@ Parse and check the configuration file
.Ar config ,
then exit.
No command is executed.
+.It Fl n
+Non interactive mode, fail if
+.Nm
+would prompt for password.
.It Fl s
Execute the shell from
.Ev SHELL
Index: doas.c
===
RCS file: /build/data/openbsd/cvs/src/usr.bin/doas/doas.c,v
retrieving revision 1.21
diff -u -p -r1.21 doas.c
--- doas.c 24 Jul 2015 06:36:42 - 1.21
+++ doas.c 26 Jul 2015 11:13:52 -
@@ -295,9 +295,10 @@ main(int argc, char **argv, char **envp)
int ngroups;
int i, ch;
int sflag = 0;
+ int nflag = 0;
uid = getuid();
- while ((ch = getopt(argc, argv, C:su:)) != -1) {
+ while ((ch = getopt(argc, argv, C:nsu:)) != -1) {
switch (ch) {
case 'C':
setresuid(uid, uid, uid);
@@ -307,6 +308,9 @@ main(int argc, char **argv, char **envp)
if (parseuid(optarg, target) != 0)
errx(1, unknown user);
break;
+ case 'n':
+ nflag = 1;
+ break;
case 's':
sflag = 1;
break;
@@ -361,7 +365,7 @@ main(int argc, char **argv, char **envp)
}
if (!(rule-options NOPASS)) {
- if (!auth_userokay(myname, NULL, NULL, NULL)) {
+ if (nflag || !auth_userokay(myname, NULL, NULL, NULL)) {
syslog(LOG_AUTHPRIV | LOG_NOTICE,
failed password for %s, myname);
fail();
Can't this be achieved with doas -C /etc/doas.conf command ... and
checking if doas will print permit nopass, as it's done in Check if
command is permitted by doas thread on hackers@? I see you want to
fail later rather than sooner, though...
Possibly. What's the point ? that's one sudo option that takes about zero
code to emulate 100%. So why not ?
Re: doas -n
On Sun, Jul 26, 2015 at 03:07:06PM -0400, Ted Unangst wrote:
Marc Espie wrote:
I don't think it falls on the side of bloat, and it's a pretty nifty option
to sudo...
@@ -361,7 +365,7 @@ main(int argc, char **argv, char **envp)
}
if (!(rule-options NOPASS)) {
- if (!auth_userokay(myname, NULL, NULL, NULL)) {
+ if (nflag || !auth_userokay(myname, NULL, NULL, NULL)) {
syslog(LOG_AUTHPRIV | LOG_NOTICE,
failed password for %s, myname);
fail();
alright, so technical nit. i don't think we want it syslogging in this case. i
would add a separate block just above.
if (nflag)
fail();
or perhaps even better:
if (nflag)
errx(1, Authorization required);
yep, good for me.
Gonna commit this later.
Re: doas -n
On Sun, Jul 26, 2015 at 12:07:44PM -0400, Ted Unangst wrote: Marc Espie wrote: I don't think it falls on the side of bloat, and it's a pretty nifty option to sudo... well, it's not just about code bloat. or even mostly about code bloat. every option added to the program is added to the man page, and then users learning how to use the program must read about each option and decide yes or no. the more options there are, the more people feel like they should study the man page even longer to find the best ones. like panning for gold. -s was added not just because it was small or nifty, but because developers said they relied on it due to muscle memory. its addition made doas better. at least tell me -n is an option you need and not just a nifty addition. I actually have code in make that tried to kill processes started thru sudo that no longer works. It won't go interactive...
Re: unwritable PKG_CACHE directory
On Mon, Jul 13, 2015 at 12:21:07PM -0600, Dale Lindskog wrote: On Mon, 13 Jul 2015, li...@wrant.com wrote: I object to the silent part... if you're trying to actually use PKG_CACHE then, having it fail silently and then discovering several GB later that oops, it didn't save anything anywhere looks like a huge mistake. But I'll try to make the error message be completely explicit. http://marc.info/?l=openbsd-cvsm=143671276012679 Thank you, Marc. http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.sbin/pkg_add/OpenBSD/PackageRepository.pm.diff?r1=1.113r2=1.114 Thanks Marc, t I disagree that pkg_add should always error outwhen PKG_CACHE is unwritable. Specifically, when pkg_add is invoked the -s option, no package is copied to the package cache even if it is writable. Unless you think that pkg_add should always be run by root? -- Dale For fuck's sake, undefining PKG_CACHE if you want to run things as non-root is easy enough. Stop talking crap. It's simple. PKG_CACHE is a parameter to pkg_add. If you run other options, like -n or -s, there is an obvious interface conflict. What should pkg_add obey ? I will err on the side of KEEPING THINGS SIMPLE. If PKG_CACHE is defined, pkg_add will obey it. Don't want PKG_CACHE ? unset it while running pkg_add with those options. That way, I don't have to put more tests in there.
Re: tame(1), like nice(1) but for permissions
On Mon, Jul 20, 2015 at 10:41:08AM -0600, Theo de Raadt wrote: On Mon, Jul 20, 2015 at 12:04:43PM -0400, Ted Unangst wrote: chroot is probably the best comparision. yes, we provide a chroot(1), but There is no chroot(1). :p practically nothing uses it. everything is instead calling chroot(2) on its own. the things that do use chroot(1) are doing so for specialized namespace reasons, not for sandboxing. I have a huge counter-example: dpb. Specifically, chroot(8) does the nice usercontext thingies that would be cumbersome to do from perl. chroot was only used as a partial example. I have the same concerns with tame(1). First, it is very premature. Secondly, TAME_EXEC is a very nasty semantic. Most importantly the purpose of tame is to allow a programmer to seperate their initial-setup from the main-loop processing. By tagging the unix feature-set into a simple effect classifications, it also guides the programming of general purpose unix tools, guiding them towards privdrop, privsep; or if they have no specific priv-slit happening, at minimum it constraints most to files-only or network-only behaviours. From the outside, a regular user is not going to know the system features and semantics that a program uses, not in a detailed fashion. tame -a firefox doesn't work. Is tame broken? We don't need that kind of grief. Sorry, should have made things clearer. I just meant that chroot was a bad comparison. I can't see any sane use of a tame(1) at the moment.
Re: tame(1), like nice(1) but for permissions
On Mon, Jul 20, 2015 at 12:04:43PM -0400, Ted Unangst wrote: chroot is probably the best comparision. yes, we provide a chroot(1), but There is no chroot(1). :p practically nothing uses it. everything is instead calling chroot(2) on its own. the things that do use chroot(1) are doing so for specialized namespace reasons, not for sandboxing. I have a huge counter-example: dpb. Specifically, chroot(8) does the nice usercontext thingies that would be cumbersome to do from perl.
Re: [patch] PkgCreate.pm make it more clear why a shared library is invalid
On Wed, Nov 11, 2015 at 05:13:45PM +0100, Adam Wolk wrote: > Hi tech@, > > I have been working recently on packaging a shared library for the > first time and hit a stumbling block yesterday. > > $ make package > `/usr/ports/pobj/libwebsockets-1.5/fake-amd64/.fake_done' is up to date. > ===> Building package for libwebsockets-1.5 > Create /usr/ports/packages/amd64/all/libwebsockets-1.5.tgz > reading plist| > Error: Invalid shared library @lib lib/libwebsockets.so.5 I'm pretty sure the naming scheme is out of place in the error message. But yeah, the message is not perfect.
Re: export directive in make
Okay, I've had a bit of time to look (again) at gmake. This patch won't make it in its current form. It does only implement a very partial subset of what gmake does, namely the export VAR=VALUE syntax. I'll have to check gmake's code more closely, because the other export forms DO expand the full line, not just the variable value. I'm pretty sure it's more dangerous to recognize export partly and not support the other forms. There's also the fact that it's not a usual bsd-make construct (introduced as a known dot form).There's usually some risk of false positives, stuff where lines may start with export but not be export directives...
pledging m4
I did partial work before if was called pledge.
Now, with fork and exec, simple pledge is easy.
There might be something down the line to explicitly allow m4 to fork
and exec, but unfortunately, the corresponding macros are used by both
sendmail and autoconf scripts, so I would say they're rather ubiquitous.
Index: main.c
===
RCS file: /cvs/src/usr.bin/m4/main.c,v
retrieving revision 1.84
diff -u -p -r1.84 main.c
--- main.c 21 Dec 2014 09:33:12 - 1.84
+++ main.c 1 Nov 2015 09:12:04 -
@@ -174,6 +174,9 @@ main(int argc, char *argv[])
int n;
char *p;
+ if (pledge("stdio proc exec rpath wpath cpath", NULL) == -1)
+ err(1, "pledge");
+
if (signal(SIGINT, SIG_IGN) != SIG_IGN)
signal(SIGINT, onintr);
patch: fix top
either that, or we have to remove (and undocument) the 'k' command.
Personally, I find it incredibly useful
Index: top.c
===
RCS file: /data/openbsd/cvs/src/usr.bin/top/top.c,v
retrieving revision 1.87
diff -u -p -r1.87 top.c
--- top.c 4 Nov 2015 21:28:27 - 1.87
+++ top.c 5 Nov 2015 09:41:21 -
@@ -328,7 +328,7 @@ main(int argc, char *argv[])
preset_argc = 0;
} while (i != 0);
- if (pledge("stdio rpath getpw tty id ps vminfo", NULL) == -1)
+ if (pledge("stdio rpath getpw tty proc id ps vminfo", NULL) == -1)
err(1, "pledge");
/* set constants for username/uid display correctly */
Re: patch: fix top
On Thu, Nov 05, 2015 at 10:54:32AM +0100, Theo Buehler wrote:
> On Thu, Nov 05, 2015 at 10:42:54AM +0100, Marc Espie wrote:
> > - if (pledge("stdio rpath getpw tty id ps vminfo", NULL) == -1)
> > + if (pledge("stdio rpath getpw tty proc id ps vminfo", NULL) == -1)
> > err(1, "pledge");
>
> I think both "proc" and "id" are too much: "id" was added to enable
> renicing, but "proc" is good enough for that:
>
> Index: usr.bin/top/top.c
> ===
> RCS file: /cvs/src/usr.bin/top/top.c,v
> retrieving revision 1.87
> diff -u -p -r1.87 top.c
> --- usr.bin/top/top.c 4 Nov 2015 21:28:27 - 1.87
> +++ usr.bin/top/top.c 5 Nov 2015 09:54:04 -
> @@ -328,7 +328,7 @@ main(int argc, char *argv[])
> preset_argc = 0;
> } while (i != 0);
>
> - if (pledge("stdio rpath getpw tty id ps vminfo", NULL) == -1)
> + if (pledge("stdio rpath getpw tty proc ps vminfo", NULL) == -1)
> err(1, "pledge");
>
> /* set constants for username/uid display correctly */
WHAT ? I don't see setpriority in the list of syscalls that proc enables.
Re: patch: fix top
On Thu, Nov 05, 2015 at 06:23:52PM +0100, Theo Buehler wrote: > On Thu, Nov 05, 2015 at 11:52:34AM -0500, Michael McConville wrote: > > > It's not documented so it doesn't exist for me. :P > > > > > > (hint hint) > > > > Seconded. > > Here is an update for some missing syscalls for > "stdio", "proc" and "id": > > Index: lib/libc/sys/pledge.2 > === > RCS file: /cvs/src/lib/libc/sys/pledge.2,v > retrieving revision 1.12 > diff -u -p -r1.12 pledge.2 > --- lib/libc/sys/pledge.2 3 Nov 2015 16:31:54 - 1.12 > +++ lib/libc/sys/pledge.2 5 Nov 2015 17:21:24 - > @@ -155,11 +155,11 @@ previously allocated file descriptors: > .Xr getresgid 2 , > .Xr getresuid 2 , > .Xr getrlimit 2 , > +.Xr getrusage 2 , > .Xr getsid 2 , > .Xr getthrid 2 , > .Xr gettimeofday 2 , > .Xr getuid 2 , > -.Xr getuid 2 , > .Xr issetugid 2 , > .Xr kevent 2 , > .Xr kqueue 2 , > @@ -174,8 +174,10 @@ previously allocated file descriptors: > .Xr pipe 2 , > .Xr pipe2 2 , > .Xr poll 2 , > +.Xr ppoll 2 , > .Xr pread 2 , > .Xr preadv 2 , > +.Xr pselect 2 , > .Xr pwrite 2 , > .Xr pwritev 2 , > .Xr read 2 , > @@ -189,9 +191,13 @@ previously allocated file descriptors: > .Xr setitimer 2 , > .Xr shutdown 2 , > .Xr sigaction 2 , > +.Xr sigaltstack 2 , > +.Xr sigpending 2 , > .Xr sigprocmask 2 , > .Xr sigreturn 2 , > +.Xr sigsuspend 2 , > .Xr socketpair 2 , > +.Xr stat 2 , > .Xr umask 2 , > .Xr wait4 2 , > .Xr write 2 , > @@ -412,8 +418,13 @@ Allows the following process relationshi > .Xr vfork 2 , > .Xr kill 2 , > .Xr setgroups 2 , > +.Xr setpgid 2 , > .Xr setresgid 2 , > -.Xr setresuid 2 . > +.Xr setresuid 2 , > +.Xr setrlimit 2 , > +.Xr setsid 2 , > +.Xr getpriority 2 , > +.Xr setpriority 2 . > .It Va "exec" > Allows a process to call > .Xr execve 2 . > @@ -457,9 +468,11 @@ process: > .Pp > .Xr setuid 2 , > .Xr seteuid 2 , > +.Xr setreuid 2 , > .Xr setresuid 2 , > .Xr setgid 2 , > .Xr setegid 2 , > +.Xr setregid 2 , > .Xr setresgid 2 , > .Xr setgroups 2 , > .Xr setlogin 2 , Looks like an improvement to me, thanks
Re: patch: fix top
On Thu, Nov 05, 2015 at 11:52:32AM +0100, Theo Buehler wrote:
> On Thu, Nov 05, 2015 at 11:33:56AM +0100, Marc Espie wrote:
> > On Thu, Nov 05, 2015 at 10:54:32AM +0100, Theo Buehler wrote:
> > > On Thu, Nov 05, 2015 at 10:42:54AM +0100, Marc Espie wrote:
> > > > - if (pledge("stdio rpath getpw tty id ps vminfo", NULL) == -1)
> > > > + if (pledge("stdio rpath getpw tty proc id ps vminfo", NULL) ==
> > > > -1)
> > > > err(1, "pledge");
> > >
> > > I think both "proc" and "id" are too much: "id" was added to enable
> > > renicing, but "proc" is good enough for that:
> > >
> > > Index: usr.bin/top/top.c
> > > ===
> > > RCS file: /cvs/src/usr.bin/top/top.c,v
> > > retrieving revision 1.87
> > > diff -u -p -r1.87 top.c
> > > --- usr.bin/top/top.c 4 Nov 2015 21:28:27 - 1.87
> > > +++ usr.bin/top/top.c 5 Nov 2015 09:54:04 -
> > > @@ -328,7 +328,7 @@ main(int argc, char *argv[])
> > > preset_argc = 0;
> > > } while (i != 0);
> > >
> > > - if (pledge("stdio rpath getpw tty id ps vminfo", NULL) == -1)
> > > + if (pledge("stdio rpath getpw tty proc ps vminfo", NULL) == -1)
> > > err(1, "pledge");
> > >
> > > /* set constants for username/uid display correctly */
> >
> > WHAT ? I don't see setpriority in the list of syscalls that proc enables.
> >
>
> That's right. Nevertheless /src/sys/kern/kern_pledge.c, line 234:
>
> 229 [SYS_setsid] = PLEDGE_PROC,
> 230
> 231 [SYS_setrlimit] = PLEDGE_PROC | PLEDGE_ID,
> 232 [SYS_getpriority] = PLEDGE_PROC | PLEDGE_ID,
> 233
> 234 [SYS_setpriority] = PLEDGE_PROC | PLEDGE_ID,
> 235
> 236 [SYS_setuid] = PLEDGE_ID,
>
> This was added to enable csh's nice builtin without adding "id" to
> csh's promises.
It's not documented so it doesn't exist for me. :P
(hint hint)
Re: unwritable PKG_CACHE directory
On Mon, Jul 06, 2015 at 07:15:06PM -0600, Dale Lindskog wrote:
It is discouraged but possible to run pkg_add(1) with -n or -s as a user
other than root. However, if pkg_add(1) does not have write permission to
$PKG_CACHE, then unclear error messages are produced. For example:
$ ls -ld $PKG_CACHE
drwxr-xr-x 2 root wheel 3072 Jul 2 12:13 /var/pkg_cache
$ pkg_add -vn gcal
pkg_add should be run as root
Update candidates: quirks-2.54 - quirks-2.54
quirks-2.54 signed on 2015-03-08T12:33:05Z
Fatal error: Ustar
[ftp://ftp.openbsd.org/pub/OpenBSD/5.7/packages/amd64/gcal-3.6.3p0.tgz][?]:
Error while reading header
at /usr/libdata/perl5/OpenBSD/Ustar.pm line 89.
One solution is for pkg_add(1) to silently omit the attempt to copy the
package to an unwritable $PKG_CACHE. Below is a diff that achieves this,
and modifies pkg_add(1)'s man page accordingly.
Index: pkg_add.1
===
RCS file: /cvs/src/usr.sbin/pkg_add/pkg_add.1,v
retrieving revision 1.132
diff -u -p -r1.132 pkg_add.1
--- pkg_add.1 16 Apr 2015 20:01:39 - 1.132
+++ pkg_add.1 7 Jul 2015 00:51:22 -
@@ -371,7 +371,7 @@ Don't actually install a package, just r
would be taken if it was.
Will still copy packages to
.Ev PKG_CACHE
-if applicable.
+if applicable, and if pkg_add has write permission to that directory.
.It Fl P Ar type
Check permissions for distribution, where
.Ar type
Index: OpenBSD/PackageRepository.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackageRepository.pm,v
retrieving revision 1.113
diff -u -p -r1.113 PackageRepository.pm
--- OpenBSD/PackageRepository.pm 4 Mar 2015 13:55:32 - 1.113
+++ OpenBSD/PackageRepository.pm 7 Jul 2015 00:51:22 -
@@ -534,7 +534,7 @@ sub open_pipe
$object-{pid2} = $pid2;
} else {
open STDERR, '', $object-{errors};
- if (defined $object-{cache_dir}) {
+ if (defined $object-{cache_dir} and -w $object-{cache_dir}) {
my $pid3 = open(my $in, -|);
$self-did_it_fork($pid3);
if ($pid3) {
I object to the silent part... if you're trying to actually use PKG_CACHE
then, having it fail silently and then discovering several GB later that oops,
it didn't save anything anywhere looks like a huge mistake.
But I'll try to make the error message be completely explicit.
Re: Using tame() in userland
have a much more stringent one for tsort.
Index: tsort.c
===
RCS file: /build/data/openbsd/cvs/src/usr.bin/tsort/tsort.c,v
retrieving revision 1.26
diff -u -p -r1.26 tsort.c
--- tsort.c 29 Jul 2015 10:42:37 - 1.26
+++ tsort.c 28 Aug 2015 08:03:59 -
@@ -28,6 +28,7 @@
#include sysexits.h
#include unistd.h
#include ohash.h
+#include sys/tame.h
/* The complexity of topological sorting is O(e), where e is the
* size of input. While reading input, vertices have to be identified,
@@ -798,6 +799,43 @@ find_longest_cycle(struct array *h, stru
#define plural(n) ((n) 1 ? s : )
+static void
+TAME(int flags, const char *wl[])
+{
+ if (tame(flags, wl) != 0)
+ err(1, untamed program);
+}
+
+static void
+compute_whitelist(int argc, char *argv[], char *args[])
+{
+ int c;
+ int i = 0;
+
+ while ((c = getopt(argc, argv, h:flqrvw)) != -1) {
+ switch(c) {
+ case 'h':
+ args[i++] = optarg;
+ /*FALLTHRU*/
+ case 'f':
+ case 'l':
+ case 'q':
+ case 'r':
+ case 'v':
+ case 'w':
+ break;
+ default:
+ usage();
+ }
+ }
+ argc -= optind;
+ argv += optind;
+ if (argc == 1)
+ args[i++] = argv[0];
+ args[i] = NULL;
+ optind = optreset = 1;
+}
+
int
main(int argc, char *argv[])
{
@@ -806,6 +844,10 @@ main(int argc, char *argv[])
warn_flag, hints_flag, verbose_flag;
unsigned intorder;
+ char *args[3];
+
+ compute_whitelist(argc, argv, args);
+ TAME(TAME_STDIO|TAME_RPATH, args);
order = 0;
reverse_flag = quiet_flag = long_flag =
@@ -863,6 +905,7 @@ main(int argc, char *argv[])
FILE *f;
f = fopen(argv[0], r);
+ TAME(TAME_STDIO, NULL);
if (f == NULL)
err(EX_NOINPUT, Can't open file %s, argv[1]);
order = read_pairs(f, pairs, reverse_flag, argv[1], order,
@@ -871,6 +914,7 @@ main(int argc, char *argv[])
break;
}
case 0:
+ TAME(TAME_STDIO, NULL);
order = read_pairs(stdin, pairs, reverse_flag, stdin,
order, hints_flag == 2);
break;
@@ -878,6 +922,7 @@ main(int argc, char *argv[])
usage();
}
+ TAME(TAME_RW, NULL);
{
struct arrayaux;/* Unrefed nodes/cycle reporting. */
struct arrayremaining;
Re: doas path
On Mon, Aug 31, 2015 at 06:27:15PM -0400, Ted Unangst wrote: > I think we can relax the path restriction if there's no restriction on > command. > Works for me. okay espie@
patch: refactoring tsort.c a bit
Well, I've changed my coding style a bit.
This rewrites the two blobs of code in main() to actually be distinct functions,
adds a find_normal_cycle to avoid going off the edge of the screen.
There's removal of exit() in there to allow the canari to squeak (or not).
Comments, okay...
Index: tsort.c
===
RCS file: /build/data/openbsd/cvs/src/usr.bin/tsort/tsort.c,v
retrieving revision 1.28
diff -u -p -r1.28 tsort.c
--- tsort.c 31 Aug 2015 09:36:02 - 1.28
+++ tsort.c 1 Sep 2015 08:27:11 -
@@ -137,6 +137,7 @@ static struct node *find_cycle_from(stru
static struct node *find_predecessor(struct array *, struct node *);
static unsigned int traverse_node(struct node *, unsigned int, struct array *);
static struct node *find_longest_cycle(struct array *, struct array *);
+static struct node *find_normal_cycle(struct array *, struct array *);
static void heap_down(struct array *, unsigned int);
static void heapify(struct array *, int);
@@ -153,6 +154,11 @@ static void *emem(void *);
#define DEBUG_TRAVERSE 0
static struct ohash_info node_info = {
offsetof(struct node, k), NULL, hash_calloc, hash_free, entry_alloc };
+static void parse_args(int, char *[], struct ohash *);
+static int tsort(struct ohash *);
+
+static int quiet_flag, long_flag,
+ warn_flag, hints_flag, verbose_flag;
int main(int, char *[]);
@@ -795,69 +801,77 @@ find_longest_cycle(struct array *h, stru
return n;
}
+static struct node *
+find_normal_cycle(struct array *h, struct array *c)
+{
+ struct node *b, *n;
+
+ if (hints_flag)
+ n = find_smallest_node(h);
+ else
+ n = find_good_cycle_break(h);
+ while ((b = find_cycle_from(n, c)) == NULL)
+ n = find_predecessor(h, n);
+ return b;
+}
+
#define plural(n) ((n) > 1 ? "s" : "")
-int
-main(int argc, char *argv[])
+static void
+parse_args(int argc, char *argv[], struct ohash *pairs)
{
- struct ohashpairs;
- int reverse_flag, quiet_flag, long_flag,
- warn_flag, hints_flag, verbose_flag;
+ int c;
unsigned intorder;
+ int reverse_flag;
order = 0;
reverse_flag = quiet_flag = long_flag =
warn_flag = hints_flag = verbose_flag = 0;
- nodes_init();
-
- {
- int c;
-
- while ((c = getopt(argc, argv, "h:flqrvw")) != -1) {
- switch(c) {
- case 'h': {
- FILE *f;
-
- f = fopen(optarg, "r");
- if (f == NULL)
- err(EX_NOINPUT, "Can't open hint file %s",
- optarg);
- order = read_hints(f, , quiet_flag,
- optarg, order);
- fclose(f);
- }
- hints_flag = 1;
- break;
- /*FALLTHRU*/
- case 'f':
- hints_flag = 2;
- break;
- case 'l':
- long_flag = 1;
- break;
- case 'q':
- quiet_flag = 1;
- break;
- case 'r':
- reverse_flag = 1;
- break;
- case 'v':
- verbose_flag = 1;
- break;
- case 'w':
- warn_flag = 1;
- break;
- default:
- usage();
- }
- }
-
- argc -= optind;
- argv += optind;
+ nodes_init(pairs);
+ while ((c = getopt(argc, argv, "h:flqrvw")) != -1) {
+ switch(c) {
+ case 'h': {
+ FILE *f;
+
+ f = fopen(optarg, "r");
+ if (f == NULL)
+ err(EX_NOINPUT, "Can't open hint file %s",
+ optarg);
+ order = read_hints(f, pairs, quiet_flag,
+ optarg, order);
+ fclose(f);
+ }
+ hints_flag = 1;
+ break;
+ /*FALLTHRU*/
+ case 'f':
+ hints_flag = 2;
+ break;
+ case 'l':
+ long_flag = 1;
+ break;
+ case 'q':
+ quiet_flag = 1;
+ break;
+ case 'r':
+ reverse_flag = 1;
+
Re: Does grep need a version?
Keep the version, whatever it is. The "check of ports" is annoying, because some of these checks may be in scripts and happen at runtime for ports. The way it usually happens is: something breaks -> porters don't have time to check too closely -> someone moves the ports to using gnu grep instead. I had rather we had as much stuff as possible working with the base system. Having --version is not a problem. Not having it will probably break things, and we will probably notice six months from now.
Re: Building flavors with dpb
On Thu, Dec 03, 2015 at 09:04:54AM +, Tati Chevron wrote: > I'm trying to build the no_ada flavour of gcc 4.9 using dpb on a > machine running 5.8-release, and it fails: > > # export FLAVOR=no_ada > # dpb -D BUILD_USER=ports -D CDROM_ONLY -D FTP_ONLY -F 0 -L /portswork/logs > -l /portswork/locks lang/gcc/4.9 > > Fatal: Unknown flavor(s) no_ada (in test/a) > (No flavors for this port). (in test/a) > *** Error 1 in / (/usr/ports/infrastructure/mk/bsd.port.mk:3567 '.BEGIN': > @exit 1) > dpb: Can't obtain vital information from the ports tree > Usage: dpb [-aceimqrRsuUvx] [-A arch] [-B chroot] [-C plist] [-f m] [-F m] > [-I pathlist] [-J p] [-j n] [-p parallel] [-P pathlist] [-h hosts] > [-L logdir] [-l lockdir] [-b log] [-M threshold] [-X pathlist] > [pathlist ...] > > Infact, trying to build other flavoured ports fails in a similar way. > > What am I doing wrong here? Everything ;p Read about fullpkgpaths. You want something like dpb lang/gcc/4.9,no_ada
Re: doas password prompt
On Wed, Dec 02, 2015 at 04:40:33AM -0500, Ted Unangst wrote:
> henning points out that if you are seven levels deep when doas asks for a
> password, it can be hard to tell who is asking for what password.
>
> modify the prompt to include the program name and user@host.
> - if (pledge("stdio rpath getpw proc exec id", NULL) == -1)
> + if (pledge("stdio rpath getpw tty proc exec id", NULL) == -1)
> err(1, "pledge");
I'm slightly surprised about this. Seems weird that the simpler function
doesn't twiddle the tty knob as well.
Re: Pull tmpfs fix from NetBSD
On Fri, Dec 11, 2015 at 10:05:14AM -0700, Bob Beck wrote:
>
>
>
> On Fri, Dec 11, 2015 at 01:09:30AM -0500, Michael McConville wrote:
> > Here's the PR:
> >
> > https://gnats.netbsd.org/50381
> >
> > And the commit:
> >
> > https://marc.info/?l=netbsd-source-changes=144694603617544=2
> >
> > We have very few local changes to tmpfs and we share the
> > KASSERT(de->td_node == NULL), so I think this applies to us.
> >
> > Thoughts? ok?
> >
> >
> > Index: sys/tmpfs/tmpfs_subr.c
> > ===
> > RCS file: /cvsroot/src/sys/fs/tmpfs/tmpfs_subr.c,v
> > retrieving revision 1.96.4.1
> > retrieving revision 1.96.4.1.2.1
> > diff -u -p -r1.96.4.1 -r1.96.4.1.2.1
> > --- sys/fs/tmpfs/tmpfs_subr.c 22 Dec 2014 02:05:08 - 1.96.4.1
> > +++ sys/fs/tmpfs/tmpfs_subr.c 8 Nov 2015 01:27:10 -
> > 1.96.4.1.2.1
> > @@ -451,6 +451,7 @@ tmpfs_alloc_dirent(tmpfs_mount_t *tmp, c
> > nde->td_namelen = len;
> > memcpy(nde->td_name, name, len);
> > nde->td_seq = TMPFS_DIRSEQ_NONE;
> > + nde->td_node = NULL; /* for asserts */
> >
> > *de = nde;
> > return 0;
> >
>
> Well, that diff won't apply directly, and in my opinion this is a safter way,
> considering
> the tmpfs code, to accomplish the same thing.
>
> Index: tmpfs/tmpfs_mem.c
> ===
> RCS file: /cvs/src/sys/tmpfs/tmpfs_mem.c,v
> retrieving revision 1.7
> diff -u -p -u -p -r1.7 tmpfs_mem.c
> --- tmpfs/tmpfs_mem.c 14 Mar 2015 03:38:52 - 1.7
> +++ tmpfs/tmpfs_mem.c 11 Dec 2015 17:01:06 -
> @@ -151,7 +151,7 @@ tmpfs_dirent_get(struct tmpfs_mount *mp)
> if (!tmpfs_mem_incr(mp, sizeof(struct tmpfs_dirent))) {
> return NULL;
> }
> - return pool_get(_dirent_pool, PR_WAITOK);
> + return pool_get(_dirent_pool, PR_ZERO|PR_WAITOK);
> }
Looks good to me. Don't see any point in any comment or extra code.
HEADS UP: important tweaks in proot(1)
By default, it now does unpopulate_light. I'm now reasonably confident the avoidance mechanisms for not deleting important stuff are good enough, so it will wipe chroots of anything that doesn't belong there. See the man page which was updated accordingly. Some fringe case scenarios may need to use preserve if they have chroot data that's not accounted for otherwise. In general, it should have little impact on current users of proot I know of, most of which actually use unpopulate* already.
Re: exp.3: remove ancient history and some markup tweaks
On Mon, May 30, 2016 at 03:34:04PM +0200, Joerg Sonnenberger wrote: > On Mon, May 30, 2016 at 02:16:20PM +0200, Theo Buehler wrote: > > It may be somewhat interesting to mention why expm1(x) = exp(x) - 1 and > > log1p(x) = log(1 + x) are provided and what their historical purpose is. > > However, as mlarkin@ put it: are any of our users of exp(3) going to > > seriously be asking themselves "hmm, is OpenBSD's exp compatible with > > BASIC on the HP-71B?" > > The wording change also changes the semantics quite a bit. The old > wording explained where the name came from and what the function does. > The new wording implies somewhat that the functions are obsolete, which > is far from true. Huh ? "computes the value ... accurately even for tiny arguments". That's quite enough as far as a function description goes. When it does matter, you're pretty much going to lose anyway. You can possibly want to add references to external texts (Art of computer programming, Numerical recipes, or any decent IEEE754 text would do... https://people.eecs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF is probably the best short reference IMO), but if people do floating point stuff without understanding the basics, the manpage isn't really going to help anyway. Losing the full historical explanation of the name is not that big a problem; the names are very much self-explanatory.
Re: W^X compliance
On Sun, May 29, 2016 at 12:00:14PM -0600, Theo de Raadt wrote: > If anyone decides to engage an upstream developer about their software > performing W^X violations, please be respectful, detailed, and calm. > The major W^X violators which remain are not simple pieces of > software, and their authors will not make improvements in this area in > a fortnight. It is going to take a lot of patience. Taking the opportunity to stress again the plague of the ports tree: copy-pasting bugs. We have some variations upon the same library that exist in at least 3 different variations, with various ways to show basically the same mode of failure. (shows up there wrt webkit and libffi at least, and probably some others, though I haven't looked too closely yet). Yes, copying/forking the same code saves time instead of talking to upstream to get what you need, but we do pay it heavily every time. This is, again, a very long term problem that will take forever to address (e.g., people discover again and again that antipattern, that packaging and forking alleviates their immediate problem, but causes a HUGE debt later on with respect to security issues). Well, let's be positive, and say that at least, the current W^X changes show with glaring clarity some examples of that anti-pattern at work... bookmark for later use with the next problem we face. :p
Re: lib/libsqlite3/src/sqlite3.h
On Tue, May 31, 2016 at 11:02:30PM +0100, Stuart Henderson wrote: > Does anyone know/remember how src/sqlite3.h is generated? By the target called "header" in the Makefile
Re: add mirror discovery to pkg_add
this is the exact same code that s currently in install.sub transposed ad perl
Re: add mirror discovery to pkg_add
Here's an expanded version of the patch.
So far, ask_list was happy with prompting, but the mirror list is slightly
large, so being able to pipe thru more comes in handy.
This means a bit of refactor: we've got state, so we can get the height
from a progressmeter (or the stub), and it's reasonably easy to tweak
state display to be able to use arbitrary fh...
This shows another logic limitation of the current code, namely that the
size detection of the display is linked to the progressmeter options, whereas
it should more or less always happen when we detect we got a tty connected,
since the width/height information is uncorrelated to whether or not we
need/want a progressmeter.
To be continued...
Index: OpenBSD/AddCreateDelete.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/AddCreateDelete.pm,v
retrieving revision 1.37
diff -u -p -r1.37 AddCreateDelete.pm
--- OpenBSD/AddCreateDelete.pm 15 Jun 2016 15:40:13 - 1.37
+++ OpenBSD/AddCreateDelete.pm 22 Jun 2016 15:55:29 -
@@ -43,6 +43,11 @@ sub progress
return $self->{progressmeter};
}
+sub height
+{
+ my $self = shift;
+ return $self->{progressmeter}->height;
+}
sub not
{
my $self = shift;
Index: OpenBSD/Interactive.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/Interactive.pm,v
retrieving revision 1.20
diff -u -p -r1.20 Interactive.pm
--- OpenBSD/Interactive.pm 30 Jan 2015 11:42:55 - 1.20
+++ OpenBSD/Interactive.pm 22 Jun 2016 15:55:29 -
@@ -35,13 +35,23 @@ sub ask_list
if ($self->{always}) {
return $values[0];
}
+ my ($fh, $pid);
+ if ($self->{state}->height <= @values + 1) {
+ $pid = open($fh, "|-", "more", "-c");
+ }
- $self->{state}->errsay('#1', $prompt);
+ $fh //= \*STDERR;
+
+ $self->{state}->fhsay($fh, '#1', $prompt);
my $i = 0;
for my $v (@values) {
- $self->{state}->errsay("#1\t#2: #3",
+ $self->{state}->fhsay($fh, "#1\t#2: #3",
$i == 0 ? "a" : "", $i, $v);
$i++;
+ }
+ if (defined $pid) {
+ close($fh);
+ waitpid $pid, 0;
}
LOOP:
$self->{state}->errprint("Your choice: ");
Index: OpenBSD/PackageLocator.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackageLocator.pm,v
retrieving revision 1.105
diff -u -p -r1.105 PackageLocator.pm
--- OpenBSD/PackageLocator.pm 30 Jan 2016 11:29:29 - 1.105
+++ OpenBSD/PackageLocator.pm 22 Jun 2016 15:55:29 -
@@ -24,6 +24,7 @@ use OpenBSD::PackageRepositoryList;
use OpenBSD::PackageRepository;
my $default_path;
+my $is_configured;
sub build_default_path
{
@@ -37,17 +38,93 @@ sub build_default_path
while (my $o = OpenBSD::PackageRepository->parse(\$v, $state)) {
$default_path->add($o);
}
+ $is_configured = 1;
return;
}
$default_path->add(OpenBSD::PackageRepository->new("./",
$state)->can_be_empty);
- return if $state->defines('NOINSTALLPATH');
+ if ($state->defines('NOINSTALLPATH')) {
+ $is_configured = 1;
+ return;
+ }
return unless defined $state->config->value('installpath');
+ $is_configured = 1;
for my $i ($state->config->value("installpath")) {
$default_path->add(OpenBSD::PackageRepository->new($i, $state));
}
}
+sub discover_mirror
+{
+ my ($self, $state) = @_;
+
+ # can't ask the user -> no mirror
+ return undef unless $state->is_interactive;
+
+
+ require OpenBSD::PackageRepository;
+ # ftp.openbsd.org == 129.128.5.191 and will remain at
+ # that address for the foreseeable future.
+ my $fake =
OpenBSD::PackageRepository->new("http://129.128.5.191/cgi-bin/;, $state);
+ # XXX
+ bless $fake, "OpenBSD::PackageRepository::Cgi";
+ my $l = $fake->list;
+ my @m = @$l;
+ my %h;
+ for my $d (@m) {
+ my $e = $d;
+ $d =~ s,^http://(.*?)(/.*?)?\s+(.*)$,$1\t$3,;
+ $e =~ s/\s+.*$//;
+ $h{$d} = $e;
+ }
+ $m[0] = "";
+ my $i = $state->ask_list("No mirror configured, choose one", @m);
+ if ($i eq "") {
+ return undef;
+ }
+ return $h{$i};
+}
+
+sub convert_to_packages
+{
+ my ($self, $url) = @_;
+ # mirror was "designed" for base releases.
+ # convert into short installpath version
+ $url =~ s,^http://(.*)/pub/OpenBSD$,$1, or
+ $url =~ s,$,/%c/packages/%a,;
+ return $url;
+}
+
+sub last_chance
+{
+ if ($is_configured) {
+ return [];
+ }
+ $is_configured = 1;
+
Re: new feature in pkg_add(1)
On Tue, Jun 21, 2016 at 10:43:07PM +0200, Patrik Lundin wrote: > The reason for doing this is that it is much faster than just blindly > trying to install a package, and does not hammer mirrors needlessly. > > Are there any plans to teach pkg_info -e about "%"? Is it even possible? Okay, just committed the exit code fix. pkg_info -Iq inst:python%3.5 will perform just as you would expect.
Re: new feature in pkg_add(1)
On Tue, Jun 21, 2016 at 10:43:07PM +0200, Patrik Lundin wrote: > However, "pkg_info -e" does not understand it: > === > # pkg_info -e python%2.7 > Invalid spec: python%2.7 > === > > I use pkg_info -e to check if a requested package is installed or > not prior to attempting to install/remove it. > > The reason for doing this is that it is much faster than just blindly > trying to install a package, and does not hammer mirrors needlessly. > > Are there any plans to teach pkg_info -e about "%"? Is it even possible? Doesn't fit within the model, but you can already do that thing in a different way, by choosing the right repository, namely inst. e.g., pkg_info inst:python2.7 ought to give you the right result. I need to make sure you get a sensible rc code though, which isn't okay right now.
add mirror discovery to pkg_add
Most of the code was already there.
This would allow pkg_add to auto-configure a mirror, for the case where
PKG_PATH was not specified and where pkg.conf does not exist.
It only triggers when a location ends up empty and when run in interactive
mode, e.g., it shouldn't interfere with local lookups.
Good idea, or awful ?
Index: OpenBSD/PackageLocator.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackageLocator.pm,v
retrieving revision 1.105
diff -u -p -r1.105 PackageLocator.pm
--- OpenBSD/PackageLocator.pm 30 Jan 2016 11:29:29 - 1.105
+++ OpenBSD/PackageLocator.pm 22 Jun 2016 13:51:40 -
@@ -24,6 +24,7 @@ use OpenBSD::PackageRepositoryList;
use OpenBSD::PackageRepository;
my $default_path;
+my $is_configured;
sub build_default_path
{
@@ -37,17 +38,91 @@ sub build_default_path
while (my $o = OpenBSD::PackageRepository->parse(\$v, $state)) {
$default_path->add($o);
}
+ $is_configured = 1;
return;
}
$default_path->add(OpenBSD::PackageRepository->new("./",
$state)->can_be_empty);
- return if $state->defines('NOINSTALLPATH');
+ if ($state->defines('NOINSTALLPATH')) {
+ $is_configured = 1;
+ return;
+ }
return unless defined $state->config->value('installpath');
+ $is_configured = 1;
for my $i ($state->config->value("installpath")) {
$default_path->add(OpenBSD::PackageRepository->new($i, $state));
}
}
+sub discover_mirror
+{
+ my ($self, $state) = @_;
+
+ # can't ask the user -> no mirror
+ return undef unless $state->is_interactive;
+
+
+ require OpenBSD::PackageRepository;
+ my $fake =
OpenBSD::PackageRepository->new("http://129.128.5.191/cgi-bin/;, $state);
+ # XXX
+ bless $fake, "OpenBSD::PackageRepository::Cgi";
+ my $l = $fake->list;
+ my @m = @$l;
+ my %h;
+ for my $d (@m) {
+ my $e = $d;
+ $d =~ s,^http://(.*?)(/.*?)?\s+(.*)$,$1\t$3,;
+ $e =~ s/\s+.*$//;
+ $h{$d} = $e;
+ }
+ $m[0] = "";
+ my $i = $state->ask_list("No mirror configured, choose one", @m);
+ if ($i eq "") {
+ return undef;
+ }
+ return $h{$i};
+}
+
+sub convert_to_packages
+{
+ my ($self, $url) = @_;
+ # mirror was "designed" for base releases.
+ # convert into short installpath version
+ $url =~ s,^http://(.*)/pub/OpenBSD$,$1, or
+ $url =~ s,$,/%c/packages/%a,;
+ return $url;
+}
+
+sub last_chance
+{
+ if ($is_configured) {
+ return [];
+ }
+ $is_configured = 1;
+ my ($self, @search) = @_;
+ my $state = pop @search;
+
+ my $url = $self->discover_mirror($state);
+ if (!defined $url) {
+ return [];
+ }
+
+ $url = $self->convert_to_packages($url);
+
+ # try setting it "permanently"
+ if (open(my $f, ">>", OpenBSD::Paths->pkgconf)) {
+ print $f "installpath += $url\n";
+ close $f;
+ } else {
+ $state->errsay("Couldn't write to #1", OpenBSD::Paths->pkgconf);
+ }
+
+ # use it for the current round anyway
+ $default_path->add(OpenBSD::PackageRepository->new($url, $state));
+
+ return $self->match_locations(@search, $state);
+}
+
sub default_path
{
if (!defined $default_path) {
@@ -107,4 +182,27 @@ sub match_locations
return $self->default_path($state)->match_locations(@search);
}
+package OpenBSD::PackageRepository::Cgi;
+our @ISA = qw(OpenBSD::PackageRepository::HTTP);
+
+# we know how to get a list, we just need to override the specific url
+# and parser
+sub get_http_list
+{
+ my ($self, $error) = @_;
+
+ require OpenBSD::Paths;
+ my $fullname =
$self->url."ftplist.cgi?path=".OpenBSD::Paths->os_directory."/".OpenBSD::Paths->machine_architecture;
+ my $l = [];
+ my $fh = $self->open_read_ftp(OpenBSD::Paths->ftp." -o - $fullname",
+ $error) or return;
+ while(<$fh>) {
+ chomp;
+ if (m/^http:\/\//) {
+ push(@$l, $_);
+ }
+ }
+ $self->close_read_ftp($fh);
+ return $l;
+}
1;
Index: OpenBSD/PackageRepositoryList.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackageRepositoryList.pm,v
retrieving revision 1.30
diff -u -p -r1.30 PackageRepositoryList.pm
--- OpenBSD/PackageRepositoryList.pm9 Jul 2015 12:57:55 - 1.30
+++ OpenBSD/PackageRepositoryList.pm22 Jun 2016 13:51:40 -
@@ -86,7 +86,7 @@ sub match_locations
return $l;
}
}
- return [];
+ return
new feature in pkg_add(1)
I was waiting for snapshots to come up with the new stuff, but it looks like amd64 will be a bit late. Someone is still hiking in the mountains... A week ago or so, I committed support for some disambiguating filter in pkg_add. This means that you can now simply install packages for ports which have several versions in the tree without having to know the exact version. For instance, pkg_add python%2.7 will install a python from the lang/python/2.7 branch. e.g., stem%branch takes all packages that match stem-* and filters them according to a word in their pkgpath, e.g., you can also use it for, say postfix%stable and other things. This is not really a change, but I've checked where I put %m and stuff support, it works in pkg.conf, it also works in PKG_PATH and on the command-line. This means that you can, for instance, write something like pkg_add http://ftp.fr.openbsd.org/%m/tcl%8.5 and this will grab tcl on the 8.5 branch from the "appropriate" directory of the froggie server, depending on your OpenBSD version and architecture.
Re: add mirror discovery to pkg_add
You guys made me think about the actual use case: noob user of OpenBSD, installs the ISO, never gets to have any pkg.conf by default. A way to handle that case would be to have non-network iso *installs* put a pkg.conf that says "hey we didn't configure anything, let's do that later". A bit a la first-time-boot. If you unconfigure things, end up with no pkg.conf, then it will never trigger. Anyhow, the code I have made me see thru a few "fun" pkg_add details I'll have to fix anyway (make ask_list able to deal with long lists in every case looks like a worthwhile pursuit always)
Re: Disable DSA_FLAG_NO_EXP_CONSTTIME, always enable constant-time behavior
The only thing I'm wondering about is if there's somebody out there who just uses the "big integer arithmetic" part of openssl, and doesn't want to go libgmp for licensing reasons. Like, if you're in it for (say) trying to break codes, having code that goes as fast as it can might be useful. Is this a valid concern ?
Re: new feature in pkg_add(1)
On Thu, Jun 23, 2016 at 06:44:19AM +0200, Patrik Lundin wrote: > On Wed, Jun 22, 2016 at 02:19:26PM +0200, Marc Espie wrote: > > On Tue, Jun 21, 2016 at 10:43:07PM +0200, Patrik Lundin wrote: > > > The reason for doing this is that it is much faster than just blindly > > > trying to install a package, and does not hammer mirrors needlessly. > > > > > > Are there any plans to teach pkg_info -e about "%"? Is it even possible? > > > > Okay, just committed the exit code fix. > > > > pkg_info -Iq inst:python%3.5 > > will perform just as you would expect. > > Awesome :). > > I will play around with this and see what I can come up with, thanks! Actually, I think there's something deeply wrong with this process. This is not Unix. The philosophy has always been: you want something, you try to do it. You do not check whether it's already there, and then do it if it's not. So the bigger problem is that there's no quick mode for pkg_add that checks very quickly on the installed system, stops if it's already there, and otherwise goes installing. Right now, you're doing the same check twice, which is not so cool. So I'll see if I can speed up pkg_add at some point for the cases things are already there.
Re: Updated man page for release.8 - with feedback
Pretty sure section 7 should now refer directly to bulk(8)
Re: next step, auto-branch determination
Here's another approach. I believe I do not need more than
@option is-branch
to actually mark branched ports... The current tree is clean enough
that the last component of the pkgpath is always enough to disambiguate
the path (even though there are some surprising choices of names).
Follows a quick script to test that theory and the results...
Index: OpenBSD/PackingElement.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackingElement.pm,v
retrieving revision 1.243
diff -u -p -r1.243 PackingElement.pm
--- OpenBSD/PackingElement.pm 3 Feb 2016 18:30:15 - 1.243
+++ OpenBSD/PackingElement.pm 25 Jun 2016 13:23:49 -
@@ -894,6 +894,8 @@ sub new
return OpenBSD::PackingElement::Firmware->new;
} elsif ($args eq 'always-update') {
return OpenBSD::PackingElement::AlwaysUpdate->new;
+ } elsif ($args eq 'is-branch') {
+ return OpenBSD::PackingElement::IsBranch->new;
} else {
die "Unknown option: $args";
}
@@ -941,6 +943,13 @@ sub category()
'always-update';
}
+package OpenBSD::PackingElement::IsBranch;
+our @ISA=qw(OpenBSD::PackingElement::UniqueOption);
+
+sub category()
+{
+ 'is-branch';
+}
# The special elements that don't end in the right place
package OpenBSD::PackingElement::ExtraInfo;
our @ISA=qw(OpenBSD::PackingElement::Unique OpenBSD::PackingElement::Comment);
Index: OpenBSD/PackingList.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackingList.pm,v
retrieving revision 1.136
diff -u -p -r1.136 PackingList.pm
--- OpenBSD/PackingList.pm 14 Jun 2016 15:41:31 - 1.136
+++ OpenBSD/PackingList.pm 25 Jun 2016 13:23:49 -
@@ -257,7 +257,7 @@ sub ExtraInfoOnly
{
my ($fh, $cont) = @_;
while (<$fh>) {
- if (m/^\@(?:name|pkgpath|comment\s+(?:subdir|pkgpath)\=)\b/o) {
+ if
(m/^\@(?:name|pkgpath|comment\s+(?:subdir|pkgpath)\=|option\s+is-branch)\b/o) {
&$cont($_);
# XXX optimization
} elsif (m/^\@(?:depend|wantlib|newgroup|newuser|cwd)\b/o) {
@@ -446,7 +446,7 @@ sub match_pkgpath
}
our @unique_categories =
-(qw(name url signer digital-signature no-default-conflict
manual-installation firmware always-update extrainfo localbase arch));
+(qw(name url signer digital-signature no-default-conflict
manual-installation firmware always-update is-branch extrainfo localbase arch));
our @list_categories =
(qw(conflict pkgpath ask-update depend
Index: OpenBSD/PkgInfo.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PkgInfo.pm,v
retrieving revision 1.38
diff -u -p -r1.38 PkgInfo.pm
--- OpenBSD/PkgInfo.pm 22 Jun 2016 12:18:21 - 1.38
+++ OpenBSD/PkgInfo.pm 25 Jun 2016 13:23:50 -
@@ -407,6 +407,28 @@ sub may_check_data
$handle->{checked} = 1;
}
+sub print_branch
+{
+ my ($self, $state, $pkg, $handle) = @_;
+ unless (defined $handle) {
+ $state->errsay("Error printing info for #1: no info ?", $pkg);
+ return;
+ }
+ require OpenBSD::PackingList;
+ require OpenBSD::PackageName;
+
+ my $plist = $handle->plist(\::PackingList::ExtraInfoOnly);
+ my $name = OpenBSD::PackageName->new_from_string($plist->pkgname);
+ my $stem = $name->{stem};
+ my $compose = $stem."--".join('-', sort keys %{$name->{flavors}});
+ if ($plist->has('is-branch')) {
+ if ($plist->fullpkgpath =~ m/\/(.*?)(,.*)?$/) {
+ $compose .= "%$1";
+ }
+ }
+ $state->say("#1", $compose);
+}
+
sub print_info
{
my ($self, $state, $pkg, $handle) = @_;
@@ -567,8 +589,8 @@ sub parse_and_run
}
};
$state->{no_exports} = 1;
- $state->handle_options('cCdfF:hIKLmPQ:qr:RsSUe:E:Ml:aAt',
- '[-AaCcdfIKLMmPqRSstUv] [-D nolock][-E filename] [-e pkg-name] ',
+ $state->handle_options('cCdfF:hIKLmPQ:qr:RsSUe:E:Ml:aAtz',
+ '[-AaCcdfIKLMmPqRSstUvz] [-D nolock][-E filename] [-e pkg-name] ',
'[-l str] [-Q query] [-r pkgspec] [pkg-name ...]');
if ($state->opt('r')) {
@@ -607,6 +629,9 @@ sub parse_and_run
$state->setopts('cdMR');
}
}
+ if ($state->opt('z')) {
+ $state->setopts('am');
+ }
if ($state->opt('Q')) {
require OpenBSD::Search;
@@ -674,7 +699,11 @@ sub parse_and_run
}
if (!$self->find_pkg($state, $pkg,
sub {
- $self->print_info($state, @_);
+ if ($state->opt('z')) {
+ $self->print_branch($state, @_);
+ } else {
+
next step, auto-branch determination
The following very simple code shows that it's possible to figure out
most of the branches (new % format) automatically based on comparison
between fullpkgname and fullpkgpath.
There are a few "false positive" (innocuous). I'm going to check soon that
I have them all.
This ought to yield a "better" way to achieve pkg_add -z, assuming I get
some kind of nicer pkg_info output (just the stem/flavor + possible branch).
Of course, this relies on pkgpath and pkgnames basically matching in every
case that actually matters...
#! /usr/bin/perl
use OpenBSD::PackageName;
open my $fh, "cd /usr/ports && make dump-vars |";
while (<$fh>) {
next unless m/(.*)\.FULLPKGNAME=(.*)/;
$fullpkgpath = $1;
$fullpkgname = OpenBSD::PackageName->new_from_string($2);
$stem = $fullpkgname->{stem};
if ($fullpkgpath =~ m/\b\Q$stem\E\/(.*?)([,\/].*)?$/) {
$branch = $1;
print "$fullpkgname->{stem}%$branch\n";
}
}
Re: next step, auto-branch determination
Scratch that, doesn't work for packages where the stem changes, e.g., GCC subpackages. Guess it will really need some kind of branch annotation... :(
Re: next step, auto-branch determination
Here is what it turns out as, -z option to pkg_info
Prints just simple stem--flavor[%branch] list of installed packages...
Only manual installs, no need to encumber it with dependencies.
Index: OpenBSD/PkgInfo.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PkgInfo.pm,v
retrieving revision 1.38
diff -u -p -r1.38 PkgInfo.pm
--- OpenBSD/PkgInfo.pm 22 Jun 2016 12:18:21 - 1.38
+++ OpenBSD/PkgInfo.pm 25 Jun 2016 11:37:38 -
@@ -407,6 +407,30 @@ sub may_check_data
$handle->{checked} = 1;
}
+sub print_branch
+{
+ my ($self, $state, $pkg, $handle) = @_;
+ unless (defined $handle) {
+ $state->errsay("Error printing info for #1: no info ?", $pkg);
+ return;
+ }
+ require OpenBSD::PackingList;
+ require OpenBSD::PackageName;
+
+ my $plist = $handle->plist(\::PackingList::ExtraInfoOnly);
+
+ my $name = OpenBSD::PackageName->new_from_string($plist->pkgname);
+ my $stem = $name->{stem};
+ my $compose = $stem."--".join('-', sort keys %{$name->{flavors}});
+ if ($plist->fullpkgpath =~ m/\b\Q$stem\E\/(.*?)([,\/].*)?$/) {
+ my $branch = $1;
+ unless ($branch eq $stem or $branch eq 'core') {
+ $compose .= "%$branch";
+ }
+ }
+ $state->say("#1", $compose);
+}
+
sub print_info
{
my ($self, $state, $pkg, $handle) = @_;
@@ -567,8 +591,8 @@ sub parse_and_run
}
};
$state->{no_exports} = 1;
- $state->handle_options('cCdfF:hIKLmPQ:qr:RsSUe:E:Ml:aAt',
- '[-AaCcdfIKLMmPqRSstUv] [-D nolock][-E filename] [-e pkg-name] ',
+ $state->handle_options('cCdfF:hIKLmPQ:qr:RsSUe:E:Ml:aAtz',
+ '[-AaCcdfIKLMmPqRSstUvz] [-D nolock][-E filename] [-e pkg-name] ',
'[-l str] [-Q query] [-r pkgspec] [pkg-name ...]');
if ($state->opt('r')) {
@@ -607,6 +631,9 @@ sub parse_and_run
$state->setopts('cdMR');
}
}
+ if ($state->opt('z')) {
+ $state->setopts('am');
+ }
if ($state->opt('Q')) {
require OpenBSD::Search;
@@ -674,7 +701,11 @@ sub parse_and_run
}
if (!$self->find_pkg($state, $pkg,
sub {
- $self->print_info($state, @_);
+ if ($state->opt('z')) {
+ $self->print_branch($state, @_);
+ } else {
+ $self->print_info($state, @_);
+ }
})) {
$exit_code = 1;
}
Re: next step, auto-branch determination
New version, after some rewrite and input by semarie
Changes to PkgInfo.
- fix a regexp bug... tests showed the issue.
- simplify the -z option code, as it can actually share most of the code
from the rest. This means pkg_info -zm for installed packages, and funnily
enough, pkg_info -zm somepackage will work.
Index: OpenBSD/PackingElement.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackingElement.pm,v
retrieving revision 1.243
diff -u -p -r1.243 PackingElement.pm
--- OpenBSD/PackingElement.pm 3 Feb 2016 18:30:15 - 1.243
+++ OpenBSD/PackingElement.pm 25 Jun 2016 14:58:49 -
@@ -894,6 +894,8 @@ sub new
return OpenBSD::PackingElement::Firmware->new;
} elsif ($args eq 'always-update') {
return OpenBSD::PackingElement::AlwaysUpdate->new;
+ } elsif ($args eq 'is-branch') {
+ return OpenBSD::PackingElement::IsBranch->new;
} else {
die "Unknown option: $args";
}
@@ -941,6 +943,13 @@ sub category()
'always-update';
}
+package OpenBSD::PackingElement::IsBranch;
+our @ISA=qw(OpenBSD::PackingElement::UniqueOption);
+
+sub category()
+{
+ 'is-branch';
+}
# The special elements that don't end in the right place
package OpenBSD::PackingElement::ExtraInfo;
our @ISA=qw(OpenBSD::PackingElement::Unique OpenBSD::PackingElement::Comment);
Index: OpenBSD/PackingList.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackingList.pm,v
retrieving revision 1.136
diff -u -p -r1.136 PackingList.pm
--- OpenBSD/PackingList.pm 14 Jun 2016 15:41:31 - 1.136
+++ OpenBSD/PackingList.pm 25 Jun 2016 14:58:49 -
@@ -257,7 +257,7 @@ sub ExtraInfoOnly
{
my ($fh, $cont) = @_;
while (<$fh>) {
- if (m/^\@(?:name|pkgpath|comment\s+(?:subdir|pkgpath)\=)\b/o) {
+ if
(m/^\@(?:name|pkgpath|comment\s+(?:subdir|pkgpath)\=|option\s+is-branch)\b/o) {
&$cont($_);
# XXX optimization
} elsif (m/^\@(?:depend|wantlib|newgroup|newuser|cwd)\b/o) {
@@ -446,7 +446,7 @@ sub match_pkgpath
}
our @unique_categories =
-(qw(name url signer digital-signature no-default-conflict
manual-installation firmware always-update extrainfo localbase arch));
+(qw(name url signer digital-signature no-default-conflict
manual-installation firmware always-update is-branch extrainfo localbase arch));
our @list_categories =
(qw(conflict pkgpath ask-update depend
Index: OpenBSD/PkgInfo.pm
===
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PkgInfo.pm,v
retrieving revision 1.38
diff -u -p -r1.38 PkgInfo.pm
--- OpenBSD/PkgInfo.pm 22 Jun 2016 12:18:21 - 1.38
+++ OpenBSD/PkgInfo.pm 25 Jun 2016 14:58:50 -
@@ -415,7 +415,18 @@ sub print_info
return;
}
my $plist;
- if ($state->opt('I')) {
+ if ($state->opt('z')) {
+ $plist = $handle->plist(\::PackingList::ExtraInfoOnly);
+ my $name = OpenBSD::PackageName->new_from_string($pkg);
+ my $stem = $name->{stem};
+ my $compose = $stem."--".join('-', sort keys
%{$name->{flavors}});
+ if ($plist->has('is-branch')) {
+ if ($plist->fullpkgpath =~ m/\/([^\/]+?)(,.*)?$/) {
+ $compose .= "%$1";
+ }
+ }
+ $state->say("#1", $compose);
+ } elsif ($state->opt('I')) {
if ($state->opt('q')) {
$state->say("#1", $pkg);
} else {
@@ -567,8 +578,8 @@ sub parse_and_run
}
};
$state->{no_exports} = 1;
- $state->handle_options('cCdfF:hIKLmPQ:qr:RsSUe:E:Ml:aAt',
- '[-AaCcdfIKLMmPqRSstUv] [-D nolock][-E filename] [-e pkg-name] ',
+ $state->handle_options('cCdfF:hIKLmPQ:qr:RsSUe:E:Ml:aAtz',
+ '[-AaCcdfIKLMmPqRSstUvz] [-D nolock][-E filename] [-e pkg-name] ',
'[-l str] [-Q query] [-r pkgspec] [pkg-name ...]');
if ($state->opt('r')) {
Re: libc: simplify devname() fallback
On Tue, Jun 28, 2016 at 09:46:30PM -0700, Philip Guenther wrote: > Inspired by espie@'s ttyname.c diff, here's a simplification of libc's > fallback-to-scanning-/dev code for devname(). Since devname() returns > the "name under /dev", this eliminates the string manipulation == win! > > Works when tested with "stat -Lf %Sr" on a symlink to a device with > /var/run/dev.db chmoded to 600. > > ok? > > Philip Guenther Looks good. One remark below > > Index: lib/libc/gen/devname.c > === > RCS file: /data/src/openbsd/src/lib/libc/gen/devname.c,v > retrieving revision 1.12 > diff -u -p -r1.12 devname.c > --- lib/libc/gen/devname.c13 Sep 2015 08:31:47 - 1.12 > +++ lib/libc/gen/devname.c28 Jun 2016 07:44:03 - > @@ -28,23 +28,20 @@ > * SUCH DAMAGE. > */ > > -#include > #include > +#include Why not keep them sorted ? The fact that stat.h will include types.h is an implementation detail
Re: pkg_add.1
On Sun, Feb 07, 2016 at 09:42:32AM -0600, joshua stein wrote: > We don't recommend FTP mirrors anymore, installing a package via a > pipe doesn't seem to work anymore, and packages have to be signed to > be installed so the advice about miscreants is not very relevant. > > installing packages thru pipes should still work. surprised it got broken. you can still install non-signed packages if you really try.
Re: pkg_add.1
On Mon, Feb 08, 2016 at 03:05:00PM -0800, patrick keshishian wrote: > On Mon, Feb 08, 2016 at 07:28:24PM +0100, Marc Espie wrote: > > On Sun, Feb 07, 2016 at 09:42:32AM -0600, joshua stein wrote: > > > We don't recommend FTP mirrors anymore, installing a package via a > > > pipe doesn't seem to work anymore, and packages have to be signed to > > > be installed so the advice about miscreants is not very relevant. > > > > > > > > installing packages thru pipes should still work. > > surprised it got broken. > > > > you can still install non-signed packages if you really try. > > I haven't build ports in a while, but does this comment mean > that if I'm building my own ports (moving forward), the > resulting packages must be signed? Nope, just be careful not mixing your packages with other stuff, and how you move them around. Signing with your personal key is not hard. Specifically, - you can use -Dunsigned to install your own unsigned packages - beware of mixing up your unsigned packages AND official packages for dependencies (e.g., the later should be signed). Note that even with -Dunsigned, packages with a signature will *still* have their signature checked for tampering. - be careful of taking care of your unsigned packages properly. Getting them around insecure networks obviously allows tampering. You can use scp:// urls to prevent that. Automatically Signing is reasonably easy. You need to generate a couple of public/private key using signify, have them be of the form XXX-pkg.sec and XXX-pkg.pub, put at least the pubkey under /etc, then set SIGNING_PARAMETERS properly in /etc/mk.conf, e.g., SIGNING_PARAMETERS=-s signify -s /etc/XXX-pkg.sec (there's no pki. the bootstrap part means getting the pub key under the correct directory manually) Admittedly, it's safe if you trust your ports building system, and you have to take extra care (as usual) to not let /etc/XXX-pkg.sec out of your sight... You can also build packages normally and sign later off-site using pkg_sign. More cumbersome, but somewhat safer... In my opinion, there's nothing in there that's not glaringly obvious if you have a background in the use of public key cryptography. We went out of our way to make the design of the package signing system fairly intuitive and the possible security trade-offs highly visible (as far as public key cryptography can be, of course).
opendev(3) tweak
Already shown to a few people, but since pledge(2) aborts on non-dev, let's
check upfront that we're of the right type.
I don't think this requires a bump. It doesn't really change the interface,
just makes it stricter.
Index: opendev.3
===
RCS file: /build/data/openbsd/cvs/src/lib/libutil/opendev.3,v
retrieving revision 1.22
diff -u -p -r1.22 opendev.3
--- opendev.3 15 Jan 2015 19:06:32 - 1.22
+++ opendev.3 10 Mar 2016 11:51:27 -
@@ -93,7 +93,9 @@ it is modified to point at the fully exp
The
.Fn opendev
return value and errors are the same as the return value and errors of
-.Xr open 2 .
+.Xr open 2 ,
+plus
+.Er EFTYPE .
.Sh SEE ALSO
.Xr open 2 ,
.Xr getrawpartition 3 ,
Index: opendev.c
===
RCS file: /build/data/openbsd/cvs/src/lib/libutil/opendev.c,v
retrieving revision 1.15
diff -u -p -r1.15 opendev.c
--- opendev.c 30 Jun 2011 15:04:58 - 1.15
+++ opendev.c 8 Mar 2016 22:30:44 -
@@ -38,6 +38,7 @@
#include
#include
#include
+#include
#include "util.h"
@@ -50,6 +51,7 @@ opendev(const char *path, int oflags, in
{
static char namebuf[PATH_MAX];
struct dk_diskmap dm;
+ struct stat st;
char *slash, *prefix;
int fd;
@@ -106,5 +108,17 @@ opendev(const char *path, int oflags, in
if (realpath)
*realpath = namebuf;
+ if (fd != -1) {
+ if (fstat(fd, ) == -1) {
+ close(fd);
+ return (-1);
+ }
+ if ((dflags & OPENDEV_BLCK) ? !S_ISBLK(st.st_mode) :
+ !S_ISCHR(st.st_mode)) {
+ close(fd);
+ errno = EFTYPE;
+ return (-1);
+ }
+ }
return (fd);
}
Re: opendev(3) tweak
On Thu, Mar 10, 2016 at 08:48:21AM -0700, Theo de Raadt wrote: > The reason for these checks is because they protect the kernel, > and they identify a program that does the wrong thing. Here, a > program did the wrong thing. I am 100% in agreement that opendev > may not be the right place to do this. That kind of stems from > the design of opendev regarding DUID conversion I think we all > knew that wasn't the best design early on, but we needed to get > that going, before the rest of the DUID subsystem could work... If it's opendev that does the check, where should it be done ? Right now krw added checks in both disklabel AND fdisk to prevent the issue from coming up. BUT both those do opendev upfront, so it's really duplicated code. The other thing that (in my opinion) makes sense would be to duplicate the way ttys are handled, e.g., we've got a isatty library call that validates an fd so that further tty(4) ioctl will work. Maybe this is what's needed, or something similar. I don't think leaving the check to the individual programs is a good solution. I just stumbled on fdisk/disklabel being slightly broken by accident... after a few months of pledge(2) testing.
Re: COLUMNS handling
On Mon, Mar 14, 2016 at 01:38:48AM -0600, Anthony J. Bentley wrote: > COLUMNS handling in our tree is inconsistent. > > POSIX specifies that if COLUMNS is a valid value (read: 1 or greater), > it takes precedence; otherwise, width is handled in an unspecified > manner. > > Most programs follow COLUMNS, TIOCGWINSZ if that fails, and use 80 if > that fails. Some do TIOCGWINSZ, then COLUMNS, then 80. Some use a > different value than 80. And they're all inconsistently documented, if > they're documented at all. > More specifically, http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html Assuming we want this, there is also LINES. We have tools that do not even look at COLUMNS and LINES in the tree. For instance, tmux, systat, top, pkg_add, dpb. So, does it make sense to put COLUMNS and SIZE forward ? I think this is the first important question to ask... (I remember having COLUMNS and LINES hardcoded in my old, old .profile around SunOS4... we can probably assume this is gone now)
Re: opendev(3) tweak
On Mon, Mar 14, 2016 at 10:19:53PM +0100, Theo Buehler wrote: > On Thu, Mar 10, 2016 at 12:52:35PM +0100, Marc Espie wrote: > > Already shown to a few people, but since pledge(2) aborts on non-dev, let's > > check upfront that we're of the right type. > > > > I don't think this requires a bump. It doesn't really change the interface, > > just makes it stricter. > > > > If we are going to fix this in userland, I think espie@'s patch is the > way to go. I like the idea that opendev(3) validates the file descriptor > it returns. > >From the hints I've got from Theo, looks like he wants to do something else entirely... don't understand what exactly... but then that's not my area.
