Re: OpenBSD ftp and libtls: how to use session resumption with -S

2022-05-08 Thread Marc Espie
On Sun, May 08, 2022 at 10:42:52AM +0200, Hiltjo Posthuma wrote:
> On Sat, May 07, 2022 at 10:13:40PM +0200, Marc Espie wrote:
> > On Fri, May 06, 2022 at 08:13:42AM -, Stuart Henderson wrote:
> > > On 2022-05-06, Theo Buehler  wrote:
> > > > While we could readily make libssl fall back to the legacy stack if
> > > > SSL_OP_NO_TICKET is disabled, I don't think this optimization outweighs
> > > > the overall benefit of TLSv1.3 - better protocol, cleaner code.
> > > 
> > > Especially when the major beneficiary of this is pkg_add when it
> > > searches for updates; the number of connections has been *hugely*
> > > reduced with the caching added recently.
> > 
> > I haven't enforced it, but https for pkg_add  makes zero sense
> > anyway: you don't gain any confidentiality, and the integrity of
> > the package is ensured by the signatures.
> > 
> > Note that https for base release makes little sense as well, apart
> > from the initial installs. Updates will also rely on signatures,
> > so all you gain from https is... exercising tls, and noticing
> > connections are slower.
> > 
> > (also: authentication is slow for old time architectures).
> > 
> > I'm still wondering what's the point of https for all this.
> > 
> 
> The actual HTTP data sent (not just the package data itself) is not 
> immediately
> visible, filterable or changed by a MiTM. They also cannot easily see which
> packages are installed or filter errata's, right?

Actually pkg_add is mostly predictable, for debug reasons, and all data
*sizes* are still very visible.

Knowing you talk to cdn.openbsd.org, just by looking at the packet sizes
you can predict what packages are getting installed/updated.



Re: OpenBSD ftp and libtls: how to use session resumption with -S

2022-05-07 Thread Marc Espie
On Fri, May 06, 2022 at 08:13:42AM -, Stuart Henderson wrote:
> On 2022-05-06, Theo Buehler  wrote:
> > While we could readily make libssl fall back to the legacy stack if
> > SSL_OP_NO_TICKET is disabled, I don't think this optimization outweighs
> > the overall benefit of TLSv1.3 - better protocol, cleaner code.
> 
> Especially when the major beneficiary of this is pkg_add when it
> searches for updates; the number of connections has been *hugely*
> reduced with the caching added recently.

I haven't enforced it, but https for pkg_add  makes zero sense
anyway: you don't gain any confidentiality, and the integrity of
the package is ensured by the signatures.

Note that https for base release makes little sense as well, apart
from the initial installs. Updates will also rely on signatures,
so all you gain from https is... exercising tls, and noticing
connections are slower.

(also: authentication is slow for old time architectures).

I'm still wondering what's the point of https for all this.



Re: Howto do "a detailed cleanup with the aid of the sysclean package"?

2022-05-06 Thread Marc Espie
Look, can we all just agree it is just a question of adding appropriate
verbiage to the port/package, sticking a huge CAVEAT on the documentation
leading to it, and go back to writing actual code ?



Re: Howto do "a detailed cleanup with the aid of the sysclean package"?

2022-05-04 Thread Marc Espie
On Wed, May 04, 2022 at 07:45:32AM -0400, Raul Miller wrote:
> On Wed, May 4, 2022 at 4:15 AM Sebastien Marie  wrote:
> > The main problem I am seeing would be maintaining such lists, and it 
> > necessary
> > means manual addition to add only "safe" files to remove (no libraries at
> > least).
> 
> Conceptually speaking, it's possible to track library dependencies,
> and it's possible to build tools for doing so, and it's possible to
> automate this. ldd already does a lot of the work here, and a
> conceptual sysclean replacement might provide toolchain and/or install
> time and/or backup time support for tracking executable locations.
> (Conceptually, this kind of tracking would use some directory
> structure to support independent updates, and would itself be subject
> to [careful] cleanup. Some small redundancy here would probably be
> fine.)

It is not that simple.

Ports has such a tool for library tracking and automatic generation of
WANTLIB.

There are lots of caveats
- it's slow,
- taking care of library paths is complicated,
- taking care of library paths is ambiguous,
- some modern software prefer dlopen.

(We have a handful of ports that have WANTLIB that are not statically detected.
Some modern stuff in qt and gnome land will tend to dlopen some specific
components like database drivers and crypto libraries, in order to be able
to use it if it's installed or not if it's not.

I *think*, from what Antoine told me, that there's even a generic framework
somewhere in gnome that tries to use this all the time instead of classic
dynamic linking).

That said, I value the information sysclean gives me... but then I clean
my system manually regularly, and most often I know exactly what I'm getting
rid of.  All the horrors stories I've seen in this discussion are related
to people trusting it blindly/automatically.



Re: pkg-readmes missing for gnome and kde?

2022-05-02 Thread Marc Espie
On Sun, May 01, 2022 at 11:31:07PM +0300, Mihai Popescu wrote:
> On Sun, May 1, 2022 at 11:17 PM Antoine Jacoutot  
> wrote:
> >
> > On Sun, May 01, 2022 at 11:06:15PM +0300, Mihai Popescu wrote:
> > > On Sun, May 1, 2022 at 9:05 PM Antoine Jacoutot  
> > > wrote:
> > > >
> > > > On Sun, 2022-05-01 at 20:51 +0300, Mihai Popescu wrote:
> > > > > Hello,
> > > > >
> > > > > I tried to enable gnome or kde after install in an openbsd snapshot 
> > > > > for amd64.
> > > > > Last time (some time ago) I know for sure there were some pkg-readmes
> > > > > for both gnome and kde start. I can't find them now, I have some
> > > > > readmes in the directory, but not for gnome nor kde.
> > > > >
> > > > > Is there some other way used to document gnome or kde start, please?
> > > > >
> > > > > Thank you.
> > > >
> > > > # pkg_add gnome
> > > >
> > > > There will be a readme under:
> > > > /usr/local/share/doc/pkg-readmes/gnome
> > > >
> > > > --
> > > > Antoine
> > >
> > > That's exactly how I installed gnome. Still no readme file. Here is what 
> > > i have:
> > > $ cd /usr/local/share/doc/pkg-readmes/
> >
> > That means gnome is not installed.
> > Check pkg_info.
> 
> It was not. I was cooking at the same time and didn't pay enough
> attention to the output.
> Issued the install command again , it seems some packages didn't come out.
> Also I sent this to misc@ instead of ports@.
> 
> p# pkg_add gnome
> quirks-5.17 signed on 2022-04-28T19:56:28Z
> Can't find sushi-41.2
> Can't find evince-42.2
> Can't install gnome-42.0p1: can't resolve sushi-41.2,evince-42.2
> Couldn't install evince-42.2 gnome-42.0p1 sushi-41.2
> 
> 
Check your mirror/wait for things to synch out.



Re: Font Path prompt in pkg_add

2022-04-29 Thread Marc Espie
On Thu, Apr 28, 2022 at 02:37:39PM -0500, Christopher Turkel wrote:
> I'm on OpenBSD AMD64 7.1, fresh install
> I noticed when adding fonts via pkg_add it no longer prints out "You may
> wish to" after installation is finished.

It's related to the evolution of X windows.

Quite a few years ago, fonts were mostly handled server-side.

Keith Packard started a large change in X fonts a few years ago.

Most modern applications deal with X fonts client side.

The rationale behind the change is that people who deal with
server-side font info will know about the way to change the
server path.

Most new font directories are related to new applications, so
those messages were irrelevant.


Side note: there's a big focus from some people in OpenBSD
to keeping tools mostly silent (as opposed to the awfully
chatty linux thingies).  Sometimes, this has disproportionate
consequences. End users do see those messages, whereas there's
a HUGE amount of churn in the tools that DON'T end in any visible
messages and has FAR MORE REACHING consequences to the quality
of those tools. ;)



Re: OpenBSD and multitasking

2022-04-27 Thread Marc Espie
On Tue, Apr 26, 2022 at 01:40:46PM +0200, Stefan Hagen wrote:
> Mihai Popescu wrote (2022-04-26 01:13 CEST):
> > I use OpenBSD amd64 snapshots on the following dmesg hardware.
> > The download rate on a browser was slow and I figured out with some
> > memory mapped partition that disk transfer rate was slow.
> > I can bear this since I'm not into large file transfer business. But
> > here is another interesting fact: each time my disk is used by some
> > file transfer, all the running applications, mostly GUI based are
> > stalling - that includes mostly chromium ( even if it is not chromium
> > that it does the disk data transfer).
> > 
> > My questions are: is something incorrectly set up on my computer,
> > regarding the multitasking?
> > I understand disk operations are slow, but may I say that kernel is
> > dragged in that slow transfer too (no DMA, no cache, etc.)?
> > Does this happens to all users, but since there are more powerful
> > configuration involved the delay is not so noticeable?
> 
> Mount your file systems with the softdep flag (described in mount(8)).
> This should bring HDD i/o to what you're used to on other operating 
> systems.

This has the unfortunate side-effect that this is not quite the official
configuration. softdep is somewhat supported but if you experience panics
related to this, good luck getting them fixed.

(I've stopped using softdep entirely on bulk machines. NFS having burps
from time to time is enough to cope with)



Re: Auto layout for disk partitions - a new user's perspective

2022-04-19 Thread Marc Espie
For people who really want to tinker with the ports system,
perhaps non intuitively, bulk(8) is the manpage that contains
the most exhaustive set of information about how you might
want to configure your system and the choices involved.



Re: [www] ports: delete note about $OpenBSD$

2022-04-07 Thread Marc Espie
On Thu, Apr 07, 2022 at 02:33:36PM +0300, Mikhail wrote:
> This advice no longer needed.
> 
> diff --git a/faq/ports/guide.html b/faq/ports/guide.html
> index 9cfe0db80..ae3d1d79c 100644
> --- a/faq/ports/guide.html
> +++ b/faq/ports/guide.html
> @@ -1303,12 +1303,6 @@ OpenBSD is strongly security-oriented.
>  You should read and understand this page's
>  security section.
>  
> -
> -Be sure to add the OpenBSD CVS tag to the Makefile.
> -The account name, version number, etc., will be filled in automatically
> -by CVS during commit, you do not need to add those or touch those lines
> -in an update.
> -
>  
>  The goal is to get all ported applications to support OpenBSD.
>  To achieve this goal, feed patches to support running on OpenBSD back to
> 
> 
Committed thx



Re: chroot for go webserver with pledge and unveil

2022-03-16 Thread Marc Espie
On Tue, Mar 15, 2022 at 11:32:19PM +0100, i...@tutanota.com wrote:
> Since Go has support for pledge and unveil, I was thinking about
> "imitating" the setup for httpd.
> 
> I basically need to run a Go webserver with access to MariaDB,
> but would like to chroot the Go webserver.
> 
> I was thinking that since Go by default doesn't run a webserver on
>  port 80 or 443, I would just spawn as www user on some higher
>  port and then use PF to redirect.

The age old practice of dropping privileges just works.

I assume go has bindings for setuid() and friends.



Re: Please put vi in base

2022-03-14 Thread Marc Espie
On Sun, Mar 13, 2022 at 05:54:03PM -0700, Jacqueline Jolicoeur wrote:
> Hi,
> 
> On Mar 12 16:39, i...@tutanota.com wrote:
> > I know I am not going to get any points for this, but I had to fix a broken 
> > OpenBSD box today that could not boot and I didn't have any network for a 
> > couple of hours, which made me stuck with ed from the boot media.
> 
> I encountered a problem just like you did. It would be nice to have vi 
> included in the install media yet the size makes it unrealistic.
> 
> > I spend hours wasting time, banging my head into the wall. Until I finally 
> > got network up on another machine and figured out a way to mount another 
> > OpenBSD disk in order to get access to vi in order to edit the files on the 
> > broken box.
> 
> This made me realize how important taking a few hours to learn and practice 
> basic edits with ed(1) was. Now is a great time to learn it.

Unless the install is badly broken, fsck on /usr, /var, mount them, set
TERM to something decent, and you got vi.



Re: What happened to www/art on CVSWeb? Why is it empty?

2022-02-10 Thread Marc Espie
On Thu, Feb 10, 2022 at 11:25:40AM -0500, Nick Holland wrote:
> On 2/10/22 6:34 AM, Kacper Wilgus wrote:
> > I tried to download some artwork from these pages:
> > 
> > https://www.openbsd.org/art1.html
> > https://www.openbsd.org/art2.html
> > https://www.openbsd.org/art3.html
> > 
> > But only the first one has an image, the rest of them give me 404
> > errors and I swear they used to be there just a year ago. And the
> > wayback machine proves this. Was it an error, or copyright issues?
> > It seems wierd it was just snapped out of existence without any warning.
> > 
> 
> art[123].html hasn't been referenced from the main page since OpenBSD 5.8
> (see the removal in version 1.686 of index.html, and they are not currently
> referenced in any page on the website other than art[123].html so I think
> it is safe to say it was not being maintained and deleted at some point.
> 
> I have no other info than it looks like the "problem" is more the
> continued existence of art[123].html more than the missing images.
> 
> Nick.
> 
> 
A quick look at the full cvs repository shows a few .jpg and QUITE a few
.gif in the Attic.

Just saying ;)



Re: pkg_add -u fails with "failed to open CA file '/etc/ssl/cert.pem': Permission denied"

2022-01-24 Thread Marc Espie
On Mon, Jan 24, 2022 at 10:21:33AM +0100, Harald Dunkel wrote:
> I highly appreciate the carefulness, but the error message doesn't
> indicate a user "_pkgfetch", nor is it mentioned on pkg_add(1).
> Please reconsider my suggestion made on 2022-01-14:
> 

Note that smtpd(8) doesn't mention all the users and processes it used
for privilege separation either.

Those are implementation details and will work out-of-the-box unless
you fiddle with parts you're not supposed to touch.


and if you touch those parts... you're supposed to know where to look and
how things work in the background.



Re: pkg_add -u fails with "failed to open CA file '/etc/ssl/cert.pem': Permission denied"

2022-01-18 Thread Marc Espie
On Tue, Jan 18, 2022 at 04:37:00PM +0100, Harald Dunkel wrote:
> On 2022-01-17 18:02:25, Marc Espie wrote:
> > 
> > Lol.
> > 
> > cert.pem only contains public certificates. Insisting on only root being
> > able to read it means you are going to run code as root which doesn't 
> > require
> > it. That seems way more unreasonable than your original assumption.
> > 
> 
> I am not arguing about the access permissions (which I screwed
> up), but I wonder why pkg_add run by root failed with EPERM?
> Actually root was the only one *permitted* to access this file.
> Thats not an error.

Because we use this nifty technique called privilege separation to alleviate 
issues
with bugs.

Most specifically, pkg_add runs as root, which has wy too many rights, so 
it 
doesn't connect to the network directly, it starts ftp(1), which runs as 
_pkgfetch,
which passes its result to signify which can check that the archive is properly 
signed
before decompressing it with the zlib and finally putting the result on your 
disk.

It's not rocket science, privilege separation has been around for over 20 years 
at
this point ;)



Re: pkg_add -u fails with "failed to open CA file '/etc/ssl/cert.pem': Permission denied"

2022-01-17 Thread Marc Espie
On 2022-01-14, Harald Dunkel  wrote:
> On 2022-01-14 10:42:56, Harald Dunkel wrote:
>> 
>> Hi folks,
>> 
>> trying to upgrade the installed packages I get
>> 
>> # pkg_add -u
>> https://cdn.openbsd.org/pub/OpenBSD/7.0/packages-stable/amd64/: TLS connect 
>> failure: failed to open CA file '/etc/ssl/cert.pem': Permission denied
>> https://cdn.openbsd.org/pub/OpenBSD/7.0/packages/amd64/: TLS connect 
>> failure: failed to open CA file '/etc/ssl/cert.pem': Permission denied
>> https://cdn.openbsd.org/pub/OpenBSD/7.0/packages/amd64/: empty
>> Couldn't find updates for bash-5.1.8 bzip2-1.0.8p0 ...
>
>   chmod a+rx /etc/ssl
>
> did the trick, but this doesn't look reasonable.

Lol.

cert.pem only contains public certificates. Insisting on only root being
able to read it means you are going to run code as root which doesn't require
it. That seems way more unreasonable than your original assumption.



Re: Install latest package without prompts on OpenBSD 7.0

2022-01-10 Thread Marc Espie
On Mon, Jan 10, 2022 at 10:54:22AM -0500, Ian Darwin wrote:
> > > > I am working on OpenBSD 7.0, x86_64. I'm trying to script an install
> > > > of developer tools I use, like GCC and Git. When I attempt to install
> > > > GCC I am prompted:
> > > > 
> > > > $ sudo pkg_add gcc g++
> > > > quirks-4.54 signed on 2022-01-09T19:08:35Z
> > > > Ambiguous: choose package for gcc
> > > > a0: 
> > > > 1: gcc-8.4.0p9
> > > > 2: gcc-11.2.0p0
> > > > 
> > > > I've looked over the man page at https://man.openbsd.org/pkg_add, but
> > > > I don't see an option to tell pkg_add to install the latest version of
> > > > the package.
> > > 
> > > Sure there is. 
> > > 
> > > Quoting the manpage:
> > > There is also an ambiguity related to ports with multiple branches.  
> > > For
> > > instance ‘pkg_add python’ is ambiguous, as there are several versions 
> > > of
> > > python in the ports tree.  So is ‘pkg_add postfix’.  The special form
> > > ‘pkgname%branch’ can be used to restrict matches to a branch matching 
> > > the
> > > pkgpath(7).
> > > 
> > > pkg_add gcc%11 g++%11
> > > will do the trick

> In the context of the original post, I think he meant a way to invoke 
> "pkg_add" and have
> it just install whatever the latest is, without having to know a priori that 
> there is a version 11.
> "Just install gcc, dammit". There are many ports that have version choices 
> and in the context
> of installing the latest of everything in a "scripted install", having to 
> either stop mid-install
> and answer such a prompt, or sort out in advance what ports exist in multiple 
> versions,
> is not what's wanted. It may be unwise, but it's what some people that do 
> scripted installs want.
> I have wished for this too, but it never bothered me enough to send a query. 
> :-)

For production (and thus automated installs) you do not want the latest.

We have branches and they should be used for that.

Granted, the gcc port naming is probably not the best ever.

But... what's your arch ? do you *really* want a gcc 11 that might not work
at all on your arch.


Sorry.   There is a lot more going on.   We could have further annotations
to let scripts decide between "most stable", "best esr", "most likely
to win a beauty contest"



Re: Install latest package without prompts on OpenBSD 7.0

2022-01-10 Thread Marc Espie
[forgot to Cc the list]

On Mon, Jan 10, 2022 at 11:36:04AM +0100, Marc Espie wrote:
> On Sun, Jan 09, 2022 at 10:03:01PM -0500, Jeffrey Walton wrote:
> > Hi Everyone,
> > 
> > I am working on OpenBSD 7.0, x86_64. I'm trying to script an install
> > of developer tools I use, like GCC and Git. When I attempt to install
> > GCC I am prompted:
> > 
> > $ sudo pkg_add gcc g++
> > quirks-4.54 signed on 2022-01-09T19:08:35Z
> > Ambiguous: choose package for gcc
> > a0: 
> > 1: gcc-8.4.0p9
> > 2: gcc-11.2.0p0
> > 
> > I've looked over the man page at https://man.openbsd.org/pkg_add, but
> > I don't see an option to tell pkg_add to install the latest version of
> > the package.
> 
> Sure there is. 
> 
> Quoting the manpage:
> There is also an ambiguity related to ports with multiple branches.  For
> instance ‘pkg_add python’ is ambiguous, as there are several versions of
> python in the ports tree.  So is ‘pkg_add postfix’.  The special form
> ‘pkgname%branch’ can be used to restrict matches to a branch matching the
> pkgpath(7).
> 
> pkg_add gcc%11 g++%11
> will do the trick
> 



Re: pkg_add issues

2022-01-03 Thread Marc Espie
On Mon, Jan 03, 2022 at 12:21:17PM -, Stuart Henderson wrote:
> On 2022-01-02, Jon Fineman  wrote:
> > I am in New Jersey. Is there a way for me to tell what the cdn was 
> > pointing to to help find the slow/sick server?
> 
> It's shown in HTTP response headers from cdn. Almost certainly
> it was pointing at a machine which only serves the cdn and you cannot
> access directly.

pkg_add explicitly shows it in the first file it retrieves.

For various reasons (consistency), we don't do cdn round-robin on each
package. The Redirect from the first connection will be automatically
re-used in the subsequent connections.



Re: Exit status of pkg_add

2021-10-21 Thread Marc Espie
On Tue, Oct 19, 2021 at 10:42:04AM +0900, Yuichiro NAITO wrote:
> Following patch changes pkg_add to return a error code,
> if a package name is wrong.
> 
> diff --git a/usr.sbin/pkg_add/OpenBSD/AddDelete.pm
> b/usr.sbin/pkg_add/OpenBSD/AddDelete.pm
> index 7a968cbf05d..39bee874ff1 100644
> --- a/usr.sbin/pkg_add/OpenBSD/AddDelete.pm
> +++ b/usr.sbin/pkg_add/OpenBSD/AddDelete.pm
> @@ -403,12 +403,13 @@ sub check_root
>  sub choose_location
>  {
>   my ($state, $name, $list, $is_quirks) = @_;
>   if (@$list == 0) {
>   if (!$is_quirks) {
>   $state->errsay("Can't find #1", $name);
> + $state->{bad}++;
>   $state->run_quirks(
>   sub {
>   my $quirks = shift;
>   $quirks->filter_obsolete([$name], $state);
>   });
>   }
> 
> Is it OK?
> 
> On 10/18/21 16:53, Yuichiro NAITO wrote:
> > Hi, I have a question about exit status of pkg_add command.
> > 
> > When I wrote a package install script which included typo in a package name
> > (of course it's my fault), the script didn't stop in spite of `set -e`.
> > 
> > Because pkg_add command returns 0 even if a package name is wrong.
> > Is this exit status intended or design policy of pkg_add command?
> > 
> > If not, I want a error status getting returned.
> > It will save my time to look for a typo or similar bug.
> > 
> > I can't see 'EXIT STATUS' section in the pkg_add manual of OpenBSD 7.0.
> > So, I e-mailed this question.
> > 

Basically, there are a few traps in pkg_add wrt exit status.

The main trap is that making pkg_add error out in cases it didn't requires
testing, because various automated situations that already exist might fail.

I will most probably fix that one, assuming there is no fallout.



Re: Keeping a personal ports branch

2021-10-15 Thread Marc Espie
On Fri, Oct 15, 2021 at 10:25:17AM -, Rubén Llorente wrote:
> Hi there!
> 
> I am wondering how does people around here keep local branches of the ports
> tree for personal use.
> 
> The reason I am asking is because I keep some patched ports which are suited
> to solve my problems, but not suited (or useful) for submission to the
> official ports tree. Ideally I would upgrade my personal ports tree alongisde
> the official one.
> 
> I suppose I could cvs import every -RELEASE of the ports into my own cvs
> repository, but ideally I would rather have something more automated.
> Besides, I am curious as to how people is doing it.

Put your own ports into mystuff...

if you think they might interest other people, see whether the WIP git repo
would be suitable for them, until they're mature enough.



Re: pkg_add still reporting incorrect actions

2021-10-12 Thread Marc Espie
On Sun, Oct 10, 2021 at 09:29:24AM -, Stuart Henderson wrote:
> On 2021-10-10, Otto Moerbeek  wrote:
> > On Sun, Oct 10, 2021 at 11:09:58AM +0300, Mihai Popescu wrote:
> >
> >> On Sat, Oct 9, 2021 at 11:55 PM Chris Bennett <
> >> cpb_m...@bennettconstruction.us> wrote:
> >> 
> >> > On Sat, Oct 09, 2021 at 09:53:46PM +0300, Mihai Popescu wrote:
> >> > > I am running amd64-current from snapshots. I am installing a lot of
> >> > > packages using pkg_add -vV pkg1 pkg2 ...
> >> > >
> >> > > I got some strange reports, see below. This is the third email about
> >> > this,
> >> > > maybe isn't it a big deal.
> >> > >
> >> >
> >> > pkg_add -Dsnap  is needed when snapshots and release are next to each
> >> > other
> >> > .
> >> > Chris
> >> >
> >> >
> >> I am not on a release kernel. This was observed on my computer on other
> >> snapshots, I have at least 2 reports on misc@.
> >> I don't know perl, so I am not able yet to figure out what is happening.
> >> But as with many other reports, sometimes it will be fixed.
> >
> > If you are running a snap kernel, it might think it is a release
> > kernel close to release. That's the whole point. Use -D snap and
> > you'll be ok.
> 
> The reported problem is nothing to do with snapshot/release, it's a strange
> ordering of the update set.
> 
> Mihai, please ask on ports@, maybe cc espie to make sure he sees it, I think
> he will know whether it's a problem or not.

I expect it's probably just the summary line being slightly out-of-date.

What's actually important is whether updates happened correctly or not, it
is fairly low on my priority list.



Re: Why is tmpfs not working on OpenBSD?

2021-09-13 Thread Marc Espie
On Wed, Sep 08, 2021 at 09:54:52AM -0700, Chris Bennett wrote:
> On Mon, Sep 06, 2021 at 12:44:59AM +, iio7 wrote:
> > > > Why isn't it removed? It is kinda "misguiding".
> > >
> > > Shucks, you must feel terrible about our decision.
> > 
> > Well, compared to the fact that you, back in 2016, wrote that,
> > "We don't spend hours of our time adding unimportant notes to that file.", 
> > concerning updating the FAQ about this, maybe
> > instead of giving these useless comments, that you apparently
> > have got plenty of time to do, you should actually provide some
> > kind of useful information somewhere!
> > 
> 
> Wow. I guess a 2500 page FAQ would be much better.
> 
> But, I do believe I have found an important issue to add to the porting
> section of the FAQ.
> Although it covers submitting a single port, it does not cover how to
> deal with submitting a larger project with 20+ submissions.
> 
> I learned the hard way that the methods I was using to submit ports for
> a larger project just didn't work for getting these looked at and getting
> the two OK's needed for new ports. Oops.

Note that the main current issue wrt OpenBSD ports is the chronical lack
of time to look at everything that gets submitted.

"perfect submissions" tend to get looked at more easily, but it is
frequent to have to nag a bit to get people to look at it, especially when
it is not the most used port in the world.

(I do follow the "okay required to new ports rule" and I have to nag to
get things in as well)

-- 
Marc



Re: Why is tmpfs not working on OpenBSD?

2021-09-06 Thread Marc Espie
On Sun, Sep 05, 2021 at 10:12:33PM +, iio7 wrote:
> > On 2021-09-05, iio7 <
> i...@protonmail.com
> > wrote:
> >> # mount -t tmpfs tmpfs /home/foo/tmp/
> >> mount_tmpfs: tmpfs on /home/foo/tmp: Operation not supported
> 
> > It isn't built into the standard kernels, disabled with this commit::
> 
> > revision 1.229
> > date: 2016/07/25 19:52:56
> > disable tmpfs because it receives zero maintainance.
> 
> Why isn't it removed? It is kinda "misguiding".
> 

There might be hope that someone who has the time would do proper 
maintenance...  



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-07-06 Thread Marc Espie
On Thu, Jun 24, 2021 at 02:56:16PM -0600, Theo de Raadt wrote:
> > I think the easiest path here is to incorporate the new upstream into a
> > port, unless someone is familiar with zlib and can cherrypick out the
> > commit(s) that resolve the issue. (I didn't find zlib in ports already.)
> 
> That is completely impossible.  It must be in base.  There are 3 copies
> in base -- userland, kernel, and bootblocks.  They must be kept in sync.

This is not even true, there is a 4th copy in perl's library.



Re: Who is responsible for ports.su? (admittedly a non-canon resource)

2021-06-15 Thread Marc Espie
I think that his approach is doomed to fail.

There are a lot of tricky parts to flavors and multipackages and 
normalization.  If you don't use the actual ports/packages framework code,
you have to figure it out all over again by yourself.

and there are lots of gremlins.

The official code is based off sqlports.

See how many commits there were to that code... especially the tree-walker
part and normalization part... that will give you an idea of everything
that must be gotten precisely right to yield good results.

(and btw, I initially wrote bogus code. I had a large debugging session
with Robert Nagy until I got the normalization part correct!)



Re: Packages/libraries in disarray after sysupgrade

2021-05-14 Thread Marc Espie
On Thu, May 13, 2021 at 10:47:11PM +, tetrahe...@danwin1210.me wrote:
> After upgrading 6.8->6.9 (stable, not current) using sysupgrade, I am
> finding it not possible to install packages via pkg_add
> 
> When I try to install something, I get a series of errors like " dependency library name>: bad major" or ": minor is too
> small"
> 
> I am assuming I need to be installing new packages with `pkg_add -U` to
> update the dependencies as needed. However, the manpage suggests this is not
> desirable.

Sometimes, base snapshots and package snapshots are slightly out of synch.
this is what happened (rapid bumps to the crypto parts in base).

If you really know what you're doing, you can often work your way around it
by filling in the gaps. :)


As far as pkg_add -U, it's been developed as a quick solution to be able
to install a new package on-the-fly without having to update 600+ packages
first (like say, you figured you're missing an important component right
before you go live at a conference).

But you end up with a "skewed" package set coming from different snapshots,
so you're relying on everyone not missing any bump and not missing a "tight"
dependency between two library versions... works most of the time but... not
guaranteed.



Re: Can't compile php from ports

2021-05-08 Thread Marc Espie
On Fri, May 07, 2021 at 11:08:00PM +, Mik J wrote:
> Hello,
> Does anyone knows why compiling php from ports systematically fails ? It's 
> been since openbsd 6.8 that it acts this way

Why do you ask this on misc@ instead of ports@ ?

Second, it actually works for all of us... so it must be something
you're doing.

> Renaming /usr/ports/pobj/php-7.4.19/fake-amd64/debug-pkg/Makefile.new to 
> Makefile
> > Extracting debug info from 
> > /usr/ports/pobj/php-7.4.19/fake-amd64/usr/local/bin/php-7.4
[...]
> Installing /usr/ports/lang/php/7.4/pkg/php74_fpm.rc as 
> /usr/ports/pobj/php-7.4.19/fake-amd64/etc/rc.d/php74_fpm
> ===>  Building package for php-7.4.19
> Create /usr/ports/packages/amd64/all/php-7.4.19.tgz
> Creating package php-7.4.19
> Creating package debug-php-7.4.19
> Link to /usr/ports/packages/amd64/ftp/php-7.4.19.tgz
> Link to /usr/ports/packages/amd64/ftp/debug-php-7.4.19.tgz
> `/usr/ports/pobj/php-7.4.19/fake-amd64/.fake_done' is up to date.
> > Extracting debug info from 
> > /usr/ports/pobj/php-7.4.19/fake-amd64/usr/local/bin/php-7.4
> Warning: no debug-info in 
> /usr/ports/pobj/php-7.4.19/fake-amd64/usr/local/bin/php-7.4
> dwz: /usr/ports/pobj/php-7.4.19/fake-amd64/usr/local/bin/.debug/php-7.4.dbg: 
> .debug_info section not present
> objcopy: 
> /usr/ports/pobj/php-7.4.19/fake-amd64/usr/local/bin/.debug/php-7.4.dbg: 
> Invalid operation


oh look, it's doing the same thing TWICE. But wait: there's a Makefile
involved.

Now why would it do that ?

It could only happen if somehow, the dates are completely wacked.

Would you by any chance use NFS for your pobj tree, or anything that
would get your clock to act strangely ?...



Re: Fwd: rethinking terminal login with security in mind

2021-05-05 Thread Marc Espie
On Wed, May 05, 2021 at 01:44:24AM +0200, Alessandro Pistocchi wrote:
> Sorry, my keyboard went crazy and the message was sent incomplete.
> 
> Continuing: normally the entry of username is immediately followed by the
> password entry.
> However, if the OS is busy for any reason between the two entries,
> character echo is still on.
> If I don't notice that, I may start typing the password before the OS stops
> echoing and so I show it
> to anybody around who cares to look.
> 
> Wouldn't it be better to have a way to turn off echoing of characters as
> soon as I entered my username,
> regardless of whether the OS is busy or not?

Not really. it's your job to pay attention. Specifically, if your OS is busy
or whatever, you just need to wait until the Password: prompt gets
displayed, because echo gets turned off *before* that prompt happens.


and the actual standard interface used won't change.

See readpassphrase(3), which does already protect you against many many
problems.



Re: Cultural underground legende Seymour Cray and his legacy

2021-04-22 Thread Marc Espie
Is this a new UMF experiment ?



Re: Documentation on OpenBSD's 3-process privsep model?

2021-03-30 Thread Marc Espie
On Tue, Mar 23, 2021 at 09:41:06AM +, Ottavio Caruso wrote:
> On 23/03/2021 05:53, misopolemiac wrote:
> > I'd appreciate some pointers to documentation or minimal examples of
> > the 3-process privilege separation model for OpenBSD's daemons.
> > Internet searches pointed to skeleton examples at
> > github.com/krwesterback/newd and github.com/krwesterback/newdctl, but
> > those repos are now dead and it's unclear how authoritative they were
> > in the first place.
> > 
> > 
> 
> Blind leading the blind here, but I think a good starting point would be
> recent presentations by Marc Espie, who, I believe but I might be wrong, is
> the developer who worked the most on privsep.
> 
> http://www.openbsd.org/events.html

Definitely not at all.

I haven't worked the most on privsep, by far.

and the examples I've worked on are highly specific and probably 
not applicable to most of the base code.



Re: GCC only on OpenBSD adds -L/usr/lib as prefix, why? Re: OpenBSD: Failing to link custom libpng to custom libz, any thoughts how fix?

2021-03-03 Thread Marc Espie
On Wed, Mar 03, 2021 at 06:10:22PM +, Bob wrote:
> Does that -L/usr/lib really need to be in the leading position???


I have zero idea how to do that purely in specs.   Have fun tinkering.

This is probably something we'll adopt but low priority.


>  * Where is GCC's default specs file say for AMD64/i386?

somewhere under /usr/lib/gcc-lib  or /usr/local/lib/gcc-lib

you can get gcc to spew it out with -dumpspecs.

>  * Using what environment variable or GCC command line argument do
>I specify an alternative one?

Oh come on, just read the man page and  /spec :)

-specs=file  is fairly prominent.



Re: GCC only on OpenBSD adds -L/usr/lib as prefix, why? Re: OpenBSD: Failing to link custom libpng to custom libz, any thoughts how fix?

2021-03-03 Thread Marc Espie


Do you have some actual reason to use gcc for that project instead of 
clang ?...

as far as -L goes you've got a lot of choices, between linking directly to
the .so, linking with --nostdlib and putting back the pieces manually.

it's been a long time since I've last looked at gcc, we've moved to clang
a few years ago for the most part. gcc is mostly there for the legacy
architectures that do not have clang support.

Oh, I remember now, it's because of ld.ldd, the linker from clang.
see, that one does not link with /usr/lib by default, which tends to break
everything.


Note that you don't have to recompile gcc to change that: the specs file
is where the magic happens, and hey, you can specify a new one on the command
line, so you just need to copy and change.

But again: why gcc ?



Re: OpenBSD: Failing to link custom libpng to custom libz, any thoughts how fix?

2021-02-24 Thread Marc Espie
On Wed, Feb 24, 2021 at 02:17:14PM -, Stuart Henderson wrote:
> On 2021-02-23, Bob  wrote:
> > Hi,
> >
> > I am trying to make a custom build of libpng in my home directory,
> > using a libz build that I made in my home directory also.
> >
> > Both are latest version, libpng 1.6.37 same as OpenBSD's port
> > https://cvsweb.openbsd.org/ports/graphics/png/Makefile?rev=1.125=text/x-cvsweb-markup
> > and libz the latest one they published. For zlib it's 1.2.11
> > from http://zlib.net/ from 2017, OpenBSD does not have a port but
> > base bundles a 2009 version. Since 2009, significantly an export
> > "inflateReset2" has been added.
> 
> As you have seen it is difficult to have library functions in one
> version in base and in another version built elsewhere (whether that's
> in $HOME or in ports). Ports only does this for libraries where there's
> really no other choice and where that has been done they're used *very*
> rarely (the port can then not depend on any libraries which pull in the
> library from base). Currently that is libbind (used only by asdig and
> zeek) and openssl (used as a static library by sslscan, and dynamic
> for nrpe and nsca-ng).
> 
> > I'll start with the detail problem, and discuss the reproduction at the
> > bottom:
> >
> >
> > This has brought me to the bizarre issue that the libpng build plainly
> > refuses to link to my custom libz file /home/myuser/lib/libz.so.1 .
> > Instead, it insists with linking to the OS' global
> > /usr/lib/libz.so.5.0 .
> 
> This is as expected really, it looks for the higher library version
> number.

Not really, it's more that you have to make sure to put your own directory
before the system directory, which requires a bit more magic than just -L.

(remember that linking will stop at the first directory with a satisfying
library, and link with the highest version number found in there, which
is fortunate for stuff like libtool!)



Re: pkg_add version scripting ?

2020-11-01 Thread Marc Espie
On Sun, Nov 01, 2020 at 12:59:22PM +, Laura Smith wrote:
> 
> 
> I did actually try "pkg_add gnupg%2" but pkg_add didn't like that.  Will go 
> try "pkg_add gnupg%gnupg2" instead

The branches are directly picked from the pkgpath in the ports tree.

It's the one case where they have an actual meaning for end user.

and there are good reasons to use them (some ports have stable and snapshot
branches)



Re: pkg_add version scripting ?

2020-11-01 Thread Marc Espie
On Sun, Nov 01, 2020 at 12:23:44PM +, Laura Smith wrote:
> Hi
> 
> As far as I can tell from the docs, only pkg_info supports spec style ?
> 
> I am trying to script an OpenBSD setup and as part of that certain packages 
> need to be installed. For most packages that is not a problem, however, for 
> example, with gnupg, there are two versions in the 6.8 repo :
> gnupg-1.4.23p4.tgz
> gnupg-2.2.23p0.tgz
> 
> I cannot seem to find a way to tell pkg_add to install >=2  (or better still, 
> install "the latest" version).

"Better still" to be discussed.

Newer is not always better, especially when some things stop working and
stuff like that.

It's generally considered that scripting is meant to give you a reliable
system...


> Any ideas ?

You should read pkg_add(1) more closely.

 pkg_add also understands ‘stems’, that is, package names without any
 version specification.  For instance, with ‘pkg_add kdelibs’, pkg_add
 will look in the current directory (or the PKG_PATH) for a kdelibs
 package.

 pkg_add may ask questions in interactive mode, or error out otherwise.
 Interactive mode is the default on a tty, see options -I/i.

 For instance ‘pkg_add screen’ is ambiguous as it matches screen-4.03p6
 and screen-4.03p6-shm.

 To avoid ambiguities, pkg_add supports ‘stems with flavors’, that is, a
 stem separated from flavors with a double dash.  For instance, the
 previous ambiguity could be resolved by using ‘pkg_add screen--’ (matches
 only the empty flavor) or ‘pkg_add screen--shm’ (matches only the shm
 flavor).

 There is also an ambiguity related to ports with multiple branches.  For
 instance ‘pkg_add python’ is ambiguous, as there are several versions of
 python in the ports tree.  So is ‘pkg_add postfix’.  The special form
 ‘pkgname%branch’ can be used to restrict matches to a branch matching the
 pkgpath(7).

 The above ambiguities can be resolved using ‘pkg_add postfix%stable’ and
 ‘pkg_add python%3.4’, respectively.


stuff like pkg_add gnupg%gnupg and pkg_add gnupg%gnupg2
in 6.8, with possibly a bit more to take care of flavors as well.

(note that pkg_info -z   will give you the exact specs you want)



Re: Using ports and updates to the release

2020-10-28 Thread Marc Espie
On Sun, Oct 11, 2020 at 09:12:13PM +0200, Ingo Schwarze wrote:
> Hi Ed,
> 
> Ed Gray wrote on Sun, Oct 11, 2020 at 07:21:32PM +0100:
> 
> > I'm still fairly new to openbsd and the idea of using ports
> > in general rather than binary packages.
> 
> You are usually better off using packages than using ports,
> especially as a new user.
> 
> Even as an experienced user doing lots of development and minor
> amounts of ports development, i use packages most of the time.

As one of the persons *responsible* for keeping the ports system
working, I do use packages all the time.

Ports are on my development setup.

The machine I write this mail from uses packages,
with about 3 ports that are just there because not committed yet.



Re: strlcpy version speed tests?

2020-07-01 Thread Marc Espie
On Wed, Jul 01, 2020 at 07:05:02AM -0500, Luke Small wrote:
> Are you clinging to traditions for some purpose? I gave two different
> versions. strlcpy3 is clearly more easily understood and even slightly
> faster and strlcpy4 which sets up the following workhorse lines which
> through timing the functions is hands down faster on my Xeon chips:
> 
> 
> strlcpy4:
> while (--nleft != 0)
>  if ((*++dst = *++src) == '\0')
> ...
> 
> the others:
> 
> while (--nleft != 0)
>   if ((*dst++ = *src++) == '\0')
> 
> ...
> 
> 
> I spoke to my favorite university computer science professor who said
> ++n is faster than n++ because the function needs to store the initial
> value, increment, then return the
> 
> stored value in the former case,
> 
> while the later merely increments, and returns the value. Apparently,
> he is still correct on modern hardware.

If you really care about speed, you should probably look into an
arch/
asm version instead



Re: How do I get the man page for a package I haven't installed yet?

2020-06-26 Thread Marc Espie
On Tue, Jun 23, 2020 at 12:20:35PM -0600, Theo de Raadt wrote:
> Ottavio Caruso  wrote:
> 
> > Hi,
> > 
> > Unless I've got it all wrong,  will only
> > display man pages for programs and commands in base. Is there a way to
> > display the man page for a package/port I haven't installed and/or
> > downloaded yet? (This assumes I haven't downloaded the ports cvs
> > tree).
> 
> Doing that would be very annoying and painful, and very few people
> would want it.  It would also substantially degrade the clarity at
> man.openbsd.org

Actually, it ought to be feasible to have the same mechanism in place for
base  as a third party mechanism.

I don't think it would be that difficult to setup, this obviously ought to
be separate from the main OpenBSD installation, as the quality of manpages
from ports is often not up-to-par compared to base.

Both Ingo and naddy and I, we've been routinely passing all manpages from
all packages through groff and mandoc and makewhatis to the point that
over 99% of them would be clean for a usage similar to man.openbsd.org



Re: New tool to (quickly) check for available package upgrades

2020-06-18 Thread Marc Espie
On Wed, Jun 17, 2020 at 09:12:08PM -, Stuart Henderson wrote:
> On 2020-06-17, Marc Espie  wrote:
> > The only way you end up with broken installations is when porters don't do
> > their jobs, that is they fail to bump a shared library or something like
> > that.
> 
> They do still break in some cases:
> 
> libA depends on libB
> someapp depends on libA, libB
> 
> libB has a major bump. libA gets updated but someapp is missed
> due to a mirror with an incomplete update. Now someapp wants old libB,
> but libA wants new libB, resulting in breakage.

Ouchie... that  one I can't do much about. :(

> (There are also situations where some installed packages are broken
> for part of a pkg_add -u run, though they do sort themselves out later -
> I forget the details but I think it was something to do with the timing
> of ldconfig runs, updates to things like glib2/pango often do this).

ldconfig is run just before actually exec'ing anything that might depend on it.

Have you run into the problem lately ?
I think it's mostly gone with @tags, since it delays running most things until
the end of pkg_add -u, so that things  have actually settled down.



Re: New tool to (quickly) check for available package upgrades

2020-06-18 Thread Marc Espie
On Wed, Jun 17, 2020 at 09:12:08PM -, Stuart Henderson wrote:
> This is already a problem when pkg_add fetches the directory listing
> (though a smaller one because the filenames don't change as often).
> 
> Firstly the contents of the mirror can change during the pkg_add run
> so the listing becomes invalid, that can happen on any mirror.
> 
> Secondly if the mirror involves a cache (CDNs) they will often cache
> the directory listing as an object - the other files in the directory
> can be out of sync with regard to the served index page. As the index is
> not too big and is hit on every pkg_add run against that mirror, it's
> highly likely to be cached, more so than probably anything else in the
> directory except the quirks package.

Yes, I've run into this one as well.   One more reason to hate CDNs

I have on my list to check about ignoring the index in some cases.
Namely: packages have the actual names of the dependencies they were built
against built-in.  It should be possible and not that hard to try grabbing
that package, even if it didn't appear in the index, if it appears to be
more recent than what the index says.



Re: New tool to (quickly) check for available package upgrades

2020-06-17 Thread Marc Espie
On Wed, Jun 17, 2020 at 09:44:32AM -0400, Jeremy O'Brien wrote:
> On Wed, Jun 17, 2020, at 08:47, Marc Espie wrote:
> > On Wed, Jun 17, 2020 at 08:28:02AM -0400, Jeremy O'Brien wrote:
> > > On Tue, Jun 16, 2020, at 21:02, Marc Espie wrote:
> > > > 
> > > > The concept you need to understand is snapshot shearing.
> > > > 
> > > > A full package snapshot is large enough that it's hard to guarantee that
> > > > you will have a full snapshot on a mirror at any point in time.
> > > > 
> > > > In fact, you will sometimes encounter a mix of two snapshots (not that 
> > > > often,
> > > > recently, but still)
> > > > 
> > > > Hence, the decision to not have a central index for all packages, but to
> > > > keep (and trust) the actual meta-info within the packages proper.
> > > > 
> > > 
> > > Sorry, I guess I should've responded to this as well. Isn't snapshot 
> > > shearing going to be a problem regardless of the existence of a single 
> > > central-index? For instance, pkg_add notices a chromium update, which 
> > > requires a newer version of a dependency that hasn't been propagated to 
> > > the mirror yet.
> > > 
> > That's not a big problem, it just stops before updating... at least
> > your installation doesn't get hosed.
> >
> 
> Sorry if I'm being dense here. I'm just trying to understand this completely.
> 
> In the current implementation, does pkg_add recursively verify every 
> dependency of a package before installing *any* of them? Or does it install 
> dependency updates as it finds them? Because if it's the latter, I can still 
> envision scenarios that break a given package, such as: Package A depends on 
> B and C. B is sync'd, C is not yet. pkg_add installs the update to B, then 
> encounters the missing C update, and bails the upgrade of A. Now you have A 
> installed which doesn't work with the new version of B. Is this something 
> that can currently happen?

Nope.  That's why we have shared libraries numbers.

We keep the old libraries around, so packages will keep working.

In the rare case where some packages have tight dependencies (see cups and 
cups-libs for instance), pkg_add will update both at the same time.

An update is done in two steps: first extract the new package(s) to temporary
files, THEN delete the old package and move the temporary files in final
location.

The only way you end up with broken installations is when porters don't do
their jobs, that is they fail to bump a shared library or something like
that.

> Even with snapshot shearing though, having this index file could provide a 
> substantial speed upgrade. Instead of having to check *all* installed 
> package's header for updates, you could use the index to know the subset of 
> packages that you expect to have actually changed, and only download *those* 
> packages' headers. If the expected "combined" sha of a given package doesn't 
> match the index's version, then the mirror is clearly out of sync and we 
> could abort an update as usual.


The problem is the multiple RTT...! if you manage to keep one single 
connection open, you get a substantial speed upgrade.

Generating a single index is more problematic than it would seem at first o
glance.

When everything goes fine, you're happy. 

If anything is out-of-synch, you're in deep shit.

This would mean having several possible modes to cope with that, we don't
have enough resources to do that.



Re: New tool to (quickly) check for available package upgrades

2020-06-17 Thread Marc Espie
On Wed, Jun 17, 2020 at 08:28:02AM -0400, Jeremy O'Brien wrote:
> On Tue, Jun 16, 2020, at 21:02, Marc Espie wrote:
> > 
> > The concept you need to understand is snapshot shearing.
> > 
> > A full package snapshot is large enough that it's hard to guarantee that
> > you will have a full snapshot on a mirror at any point in time.
> > 
> > In fact, you will sometimes encounter a mix of two snapshots (not that 
> > often,
> > recently, but still)
> > 
> > Hence, the decision to not have a central index for all packages, but to
> > keep (and trust) the actual meta-info within the packages proper.
> > 
> 
> Sorry, I guess I should've responded to this as well. Isn't snapshot shearing 
> going to be a problem regardless of the existence of a single central-index? 
> For instance, pkg_add notices a chromium update, which requires a newer 
> version of a dependency that hasn't been propagated to the mirror yet.
> 
That's not a big problem, it just stops before updating... at least
your installation doesn't get hosed.



Re: New tool to (quickly) check for available package upgrades

2020-06-16 Thread Marc Espie
On Tue, Jun 16, 2020 at 04:59:07PM -0400, Jeremy O'Brien wrote:
> Hey misc@,
> 
> I wrote a quick little tool here: 
> https://github.com/neutralinsomniac/obsdpkgup in Go to show available package 
> upgrades from your configured mirror.
> 
> It takes no more than a few seconds (the time it takes to download index.txt 
> from the package repo) to show you all packages that have received a version 
> bump. This tool *won't* show same-version package-rebuild upgrades, so it 
> shouldn't be used as a complete replacement to running 'pkg_add -u', but 
> rather as a companion to show when actual newer versions of packages are 
> released. I just noticed that in my 99% case, I was waiting anywhere from 
> 5-10 minutes for 'pkg_add -u' to complete checking all ~400 of my installed 
> packages, and it uses a considerable amount of bandwidth while doing so.
> 
> As I understand it, the pkgtools detect same-version rebuilds by downloading 
> enough of every installed package tgz to check the metadata contained within 
> to determine if an upgrade is needed. If anyone knows of an alternative way 
> to determine when a same-version package install is required, I would love to 
> know of it. In the meantime, I hope someone else can make use of this tool as 
> well.

The concept you need to understand is snapshot shearing.

A full package snapshot is large enough that it's hard to guarantee that
you will have a full snapshot on a mirror at any point in time.

In fact, you will sometimes encounter a mix of two snapshots (not that often,
recently, but still)

Hence, the decision to not have a central index for all packages, but to
keep (and trust) the actual meta-info within the packages proper.

The only way to avoid that would be to have specific tools for mirroring,
and to ask mirror sites to set aside twice as much room so that they
can rotate snapshots.

Now, each package has all dependency information in the packing-list...
which allows pkg_add to know whether to update or not.   We do a somewhat
strict check, we have tried to relax the check at some point in the past,
but this was causing more trouble than it was worth.

The amount of data transmitted during pkg_add -u  mostly depends on signify:
you have to grab at least  a full signed block, which is slightly over 64KB.

On most modern systems, the bandwidth is not an issue, the number of RTT
is more problematic.   The way to speed pkg_add -u up would be to keep the
connection alive, which means ditching ftp(1) and switching to specific code
that talks modern http together with byte-ranges.



Re: __printflike macro on OpenBSD

2020-06-11 Thread Marc Espie
On Thu, Jun 11, 2020 at 06:22:55PM +, sensiblehue wrote:
> On Thu, Jun 11, 2020 at 03:08:01PM +0200, Marc Espie wrote:
> > On Thu, Jun 11, 2020 at 04:37:34AM +, sensiblehue wrote:
> > > Hello,
> > > I was wondering why OpenBSD doesn't have a `__printflike' macro in
> > > ? FreeBSD, NetBSD, and DragonflyBSD have it and it's also
> > > available from libbsd on Linux.
> > > Personally I think it's cleaner and just as portable if not more
> > > portable, because some compilers don't support `__attribute__'.
> > 
> > 
> > What compilers ?
> 
> GCC < v2.5 and non GNU C compilers, though I see now that OpenBSD
> defines it to nothing in that case. What could be an issue is that the
> `format' attribute appeared in GCC v2.7, and invalid attributes
> generate a warning.

clang has full support for attributes, so your statement is somewhat false.




Re: __printflike macro on OpenBSD

2020-06-11 Thread Marc Espie
On Thu, Jun 11, 2020 at 04:37:34AM +, sensiblehue wrote:
> Hello,
> I was wondering why OpenBSD doesn't have a `__printflike' macro in
> ? FreeBSD, NetBSD, and DragonflyBSD have it and it's also
> available from libbsd on Linux.
> Personally I think it's cleaner and just as portable if not more
> portable, because some compilers don't support `__attribute__'.


What compilers ?



Re: writing aucat output

2020-06-05 Thread Marc Espie
On Fri, Jun 05, 2020 at 01:02:18PM +0200, Peter J. Philipp wrote:
> On Fri, Jun 05, 2020 at 12:50:53PM +0200, Marc Espie wrote:
> > On Fri, Jun 05, 2020 at 12:06:54PM +0200, Peter J. Philipp wrote:
> > > Hi,
> > > 
> > > I'm wondering how I can write to stdout on aucat?  Here is what I have:
> > > 
> > > beta$ /usr/bin/aucat -r 44100 -h wav -i ewhist2.wav -o - | hexdump -C
> > > stdout: failed to seek back to header
> > > beta$ /usr/bin/aucat -r 44100 -h wav -i ewhist2.wav -o /dev/stdout | 
> > > hexdump -
> > > /dev/stdout: failed to seek back to header
> > 
> > That's a bug/limitation on aucat. It tries to seek on stdout right after
> > setting it up, which is absurd.
> > 
> > I'll have a look later, should be very easy to fix!
> 
> Thanks...

I spoke too soon, it's way more complicated because the wav header has
to know how large the file will be... which means you would have to parse
the input file and figure out the resulting size before writing anything
out, so it's way more code than I hoped for.

:(



Re: writing aucat output

2020-06-05 Thread Marc Espie
On Fri, Jun 05, 2020 at 12:06:54PM +0200, Peter J. Philipp wrote:
> Hi,
> 
> I'm wondering how I can write to stdout on aucat?  Here is what I have:
> 
> beta$ /usr/bin/aucat -r 44100 -h wav -i ewhist2.wav -o - | hexdump -C
> stdout: failed to seek back to header
> beta$ /usr/bin/aucat -r 44100 -h wav -i ewhist2.wav -o /dev/stdout | hexdump -
> /dev/stdout: failed to seek back to header

That's a bug/limitation on aucat. It tries to seek on stdout right after
setting it up, which is absurd.

I'll have a look later, should be very easy to fix!



Re: Forgetting pkg_add -u after sysupgrade can cause ansible msyscall errors

2020-05-31 Thread Marc Espie
On Sun, May 31, 2020 at 08:19:18PM +0200, Jurjen Oskam wrote:
> Hi,
> 
> For the sake of the archives and future search engine users, I'll share what
> can happen when you use sysupgrade to upgrade your OpenBSD host but then
> forget to run update your installed packages. (Yep, silly mistake, I know...)
> 
> After upgrading my Ansible host to 6.7, I noticed that each ansible command
> outputted the following error, but it continued normally after that:
> 
> msyscall 1ba41d145000 a5000 error
> 
> The second word varied with each invocation, and I did not notice any other
> adverse effects on that box. The error occurred with all ansible commands,
> such as ansible, ansible-playbook and ansible-connection.
> 
> I was >< this close to sending a help request to this list, but I realized
> I had not tried to see what would happen on a freshly installed 6.7 box.
> It was then that I noticed that the new box had ansible-2.9.7 and
> python-3.7.7, while the upgraded box had ansible-2.7.9 and python-3.6.8p0.
> This caused me to realize I had forgotten to update my packages after doing
> the sysupgrade... A quick "pkg_add -u" later and my problem was gone. D'oh!
> 
> So, moral of the story: don't forget to update your packages after a
> sysupgrade.

Heh

Always a good idea.

msyscall was kind of a "flag day". In case you don't know, most anything
dynamic that uses syscalls has to  go through a specific area in libc now,
it makes all kinds of attacks more complex.

Some of us running current went through similar burps in case we forgot.

I must say, I'm happy I added @tag support a while back, this made this
specific package update much less painful.



Re: Article OpenBSD: Not Free Not Fuctional and Definetly Not Secure and BSD, the truth blog

2020-05-28 Thread Marc Espie
On Thu, May 28, 2020 at 01:15:36PM -0700, Amarendra Godbole wrote:
> Aha! So my hunch was right -- I thought that'd be the case seeing the
> comments under your name that were totally out of character from your
> posts here.
> 
> -ag
> 
> On Thu, May 28, 2020 at 12:08 PM Kevin Chadwick  wrote:
> >
> > On 2020-05-28 18:38, Amarendra Godbole wrote:
> > > It indeed is written by someone lacking knowledge about everything. It
> > > is funny, and gave me a good laugh - the comments are even funnier!
> >
> > Be aware that the author deletes your comments and replaces them with his 
> > own,
> > under your name, whilst hiding behind wordpress.com!
> >
> 
> 
Actually that scum is hosted at wordpress.com, I'm not sure that they
would like knowing someone is changing comments without changing the
authors' names.

Also, that shit is anonymous, meaning the bastard can't even take
responsability for his drivel



Re: Article OpenBSD: Not Free Not Fuctional and Definetly Not Secure and BSD, the truth blog

2020-05-28 Thread Marc Espie
On Thu, May 28, 2020 at 07:58:45PM +, Kevin Chadwick wrote:
> On 2020-05-28 18:38, Amarendra Godbole wrote:
> > It indeed is written by someone lacking knowledge about everything. It
> > is funny, and gave me a good laugh - the comments are even funnier!
> 
> Be aware that the author deletes your comments and replaces them with his own,
> under your name, whilst hiding behind wordpress.com!
> 
> 
Ah I saw your name and was wondering.

So that guy is a pure asshole



Re: Article OpenBSD: Not Free Not Fuctional and Definetly Not Secure and BSD, the truth blog

2020-05-28 Thread Marc Espie
On Thu, May 28, 2020 at 01:16:59AM -0300, Quantum Robin wrote:
> Hi,
> 
> While surfing on the Google to learn more about OpenBSD, I encountered this
> one: "OpenBSD: Not Free Not Fuctional and Definetly Not Secure (
> https://aboutthebsds.wordpress.com/2013/01/25/20/)
> 
> Is the author telling the truth? Or just yet another anti-BSD thing?
> 

"At meetings, people are often physically attacked for having even a minor 
disagreement with de Raadt"

Hyperbole much ?

Theo has been known to be fairly opiniated, but "physically attacked"?

How can you take this guy seriously ?



Re: Problem with pkg_add -uv after 6.7 upgrade on i386

2020-05-25 Thread Marc Espie
On Mon, May 25, 2020 at 01:01:07PM +0200, Paolo Aglialoro wrote:
> Hello Folks,
> 
> I just upgraded a PIII box freshly installed with 6.6 last month.
> Everything went right with sysupgrade (big kudos do devs).
> 
> Problems started when upgrading installed packages, here follows the output
> of pkg_add -uv. What can this be?

Corruption of your filesystem *prior* to running pkg_add -u

Run pkg_check.

> adwaita-icon-theme-3.34.3:.libs-pango+.libs1-pango+.libs2-pango+.libs3-pango+.libs4-pango+pango-1.42.4p3->pango-1.44.7p0
> 't read packing-list from installed package
> \00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00
> Invalid \0 character in pathname for ftis: /var/db/pkg/\0 at
> /usr/libdata/perl5/OpenBSD/PackingList.pm line 520.
> File
> /var/db/pkg/\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00/+CONTENTS
...

those are actually fairly obvious error messages.



Re: Why does OpenBSD still include Perl in its base installation?

2020-05-25 Thread Marc Espie
Another thing to consider: why is perl in the base system.

Assume you need a script language, because writing everything in C is
cumbersome.

What are the choices ?
- you need something under and acceptable licence, so python is out.
(Artistic Licence is "close enough");
- you need something that builds everywhere, so python is out (hard to build
without dynamic libraries, that was vax...);
- you want a modicum of security, so shell and tcl and php are out.
- awk would kind of work, except it's not that readable, and it wouldn't
scale up to some of the things we use perl for.

Perl constituted a great compromise back in the day I chose it to replace
pkg_add. It still IS a great compromise, and it's not THAT large compared
to other pieces.

Over the years, things have moved back from language to language.
makewhatis used to be C+shell, then it became perl, and now it's pure C
because it's integrated in mandoc... I shudder to think how much time
was spent in there.

Note that Ingo also moved /etc/security from shell to perl, so he's not
adverse to perl.

As far as I'm concerned, having perl in the base system is a strength of
OpenBSD... it does minimize the number of script languages used for
ports infrastructure as compared to other languages.

perl also has impressive support tools, be they in the base system or in
ports.   NYTProf  is still the  best profiler I've ever seen on any
language.



Re: OpenBSD sysupgrade rocks

2020-05-20 Thread Marc Espie
On Tue, May 19, 2020 at 07:06:05AM +, Frank Beuth wrote:
> On Wed, May 20, 2020 at 02:07:27PM -0400, Chris Bennett wrote:
> > Please don't beg for features.
> > That's very irritating and wastes everyone's time.
> > 
> > Please don't ask for features, once again.
> > Really, I mean it. Don't ask for features!
> 
> How about a counterpart to `sendbug` called `requestfeature`, which is an
> alias to `fortune` run with a file consisting of Theo's snarkiest remarks?


Theo's snarkiest remarks are definitely NSFW!



Re: Why isn't src included with OpenBSD? (documentation)

2020-05-19 Thread Marc Espie
On Mon, May 18, 2020 at 08:43:19PM +0100, Ottavio Caruso wrote:
> Some of these documents have a proprietary licence attached to it and
> I believe it's due to the 1994 AT settlement. There are third party
> collections (like this: https://github.com/sergev/4.4BSD-Lite2) but
> I'm not sure if one could import them all in the source tree or in the
> ports tree without violating some copyright here and there.

If the README.md is accurate, that could be imported in the ports tree,
but there would be a lot of work to extract useful stuff from there.

Looks like a standard 4-clauses old-style BSD licence.

I've had a look, and it is very strange, some of the PSD documents have
been converted to pdf, BUT not all of them.

There are also no actual releases, so you'd have to pull a specific tag
from github, always hasardous...



Re: Why isn't src included with OpenBSD? (documentation)

2020-05-18 Thread Marc Espie
On Mon, May 18, 2020 at 01:07:36PM -0400, Andras Farkas wrote:
> I saw in fsck_ffs.8
> https://man.openbsd.org/fsck_ffs.8
> that the answers could be found in
> Fsck_ffs - The UNIX File System Check Program
> This is perfectly fine.  Not every piece of information belongs in a
> man page.  Man pages are the right format for some sorts of info, and
> absolutely the wrong format for some other sorts.
> BUT: I looked and couldn't find it, and ended up using
> https://docs.freebsd.org/44doc/smm/03.fsck/paper.pdf
> which is where I found my answer.
> Only after I already solved the problem did I find that the mentioned
> file exists here:
> https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sbin/fsck_ffs/SMM.doc/

Most of these files haven't been updated in ages.

The few remaining in the source tree are there as reference for people
when writing documentation and updating the source.

Each time we see them, we cringe, and keep tidying the source code and
the man page.

Speaking for personal experience working on make.

If you really want the source code for documentation purposes, what prevents
you from getting it off cvs  or a github mirror ?

Seriously, *none* of those files are necessary *for beginners*. Once you
reach the stage where you might benefit from them (say, because you actually
want to become a developer, and could learn from worse sources), you should
be able to figure out how to get them.

Having an extra set here means... more options... more code that can go
wrong, and thus a SLOWER turn-around for snapshots, which is definitely
a bad thing.



Re: fw_update verify firmware?

2020-05-14 Thread Marc Espie
On Thu, May 14, 2020 at 04:25:11AM +, Mogens Jensen wrote:
> I was just trying out the fw_update program on OpenBSD 6.5, deleting/
> installing all the firmware and was wondering if fw_update will verify
> the files before installing?

Others pointed out that firmwares are signed.

For a while now, fw_update AND pkg_add have defaulted to requiring
signed packages. If you try to install or even peek at something
unsigned with pkg_info, they error out.

You have to explicitly ask for unsigned data to bypass the check.



Re: pkg_add can't resolve package - bad major

2020-05-04 Thread Marc Espie
On Sun, May 03, 2020 at 12:58:41PM -0400, Chris Bennett wrote:
> I have had this exact same problem before
> 
> pkg_info -q > packages_installed
> pkg_delete gettext.
> pkg_add gettext-runtime
> pkg_add -u
> pkg_add -zl packages_installed
> 
Update your procedures, use pkg_info -z and not pkg_add -z.
It's been there for ages.



Re: List a package's dependencies

2020-04-20 Thread Marc Espie
On Mon, Apr 20, 2020 at 07:45:37PM +0100, Kevin Chadwick wrote:
> 
> > There are some unavoidable complexities to the sheer size of the tree,
> > and the necessities of updates not to fail...
> 
> I have noticed recently that I occasionally get a gz truncated message (I 
> think
> due to tcp timeout) and then the dependent package doesn't get updated. I then
> re-run pkg_add -u.
> 
> Is the way to deal with this from a script, to; re-run pkg_add -u, if it's
> output is > 0 ?
Nope, it's definitely the wrong place to fix things.

You should fix your pipes (change the timeouts or whatever).

If worse comes to worst, pkg_add could *possibly* retry running ftp(1),
but that makes little sense. If the transport layer is *broken* my gut
feeling is that the transport layer needs fixing! Application layer should
be able to rely on a *working* transport layer.

That's what all network books have been saying for >30 years.



Re: List a package's dependencies

2020-04-20 Thread Marc Espie
On Mon, Apr 20, 2020 at 02:48:20PM +0100, Chris Rawnsley wrote:
> > BTW, any supplementary tool that does similar things directly in shell
> > has exactly zero chance to be included in the distribution.
> 
> Acknowledged. I put it out there for those that might find it useful
> but was not expecting it to be included.
> 
> 
> > The offical parser for PackingList *is* the perl code, dependency handling
> > as well.
> > 
> > Anything else is very likely to miss special cases and break atrociously.
> 
> As I am sure is the case in my shell scripting :)
> 
> Would a patch for such a feature, if it were using the proper Perl
> machinery, be considered? Or is this something that you absolutely
> do not wish to included at this time?

The main problem is the user interface: there are already a lot of things
in pkg_info, those are not really consistent, etc.

The code is already there from the most part, there *is* a compute_closure
method in RequiredBy/Requiring.

it's almost a one-liner:

#! /usr/bin/perl
use OpenBSD::RequiredBy;

print join(' ', sort OpenBSD::Requiring->compute_closure(shift)), "\n";


all the stem back-and-forth is also available.



Re: List a package's dependencies

2020-04-20 Thread Marc Espie
On Mon, Apr 20, 2020 at 02:48:20PM +0100, Chris Rawnsley wrote:
> Hi Marc,
> 
> On Mon, 20 Apr 2020, at 14:05, Marc Espie wrote:
> > Actually, not having recursive depends easily available on an installed
> > package base is  somewhat tedu-ish.
> > 
> > Most specifically, it's not very useful for anything in common usage.  What
> > would you want it for ?...  sure it's nice information, but in practice,
> > unused dependencies from former ports get gc'd with pkg_delete -a...
> 
> The use case was that I was installing Vim and it has a few variants.
> In the past I discovered that using no_x11 means that Vim does not
> automatically interface with the X11 clipboards. Makes sense but
> as I only use Vim in the terminal I do not need the GUI. Still,
> using packages is more convenient than compiling it myself so the
> question arose which variant has the least impact on my system and
> still gets me the X11 clipboard features that I desire?

Go outside of vim proper and run autocutsel, it might give you the solution.



Re: Keeping distfiles actual with port tree and cleaning old distfiles from storage automatically

2020-04-20 Thread Marc Espie
On Mon, Apr 20, 2020 at 09:45:41AM +, Martin wrote:
> I'm looking for a way to keep distfiles up-to-date locally with auto remove 
> 'old' ones in sync with actual ports tree.

dpb + clean-old-distfiles

even if you don't build/fetch with dpb, dpb -DHISTORY_ONLY
will do exactly what you want.



Re: List a package's dependencies

2020-04-20 Thread Marc Espie
On Sun, Apr 19, 2020 at 04:36:48PM +0200, Ingo Schwarze wrote:
> All this is kind of typical for the pkg tools: one question typically
> allows several different answers.  There typically isn't one single,
> canonical way of doing something.  There typically isn't one unified
> output format, but several different ways to represent information
> in the output.  Part of that is due to the unavoidable complexity
> of the system.  Other parts may be influenced by the fact that
> espie@ is not tedu@.

I don't think tedu would do much better... or we would have a ports tree
with only the 100 ports he's using, and nothing more.

There are some unavoidable complexities to the sheer size of the tree,
and the necessities of updates not to fail...

... plus the fact that the ports tree needs to stay somewhat "entry-level".


Actually, not having recursive depends easily available on an installed
package base is  somewhat tedu-ish.

Most specifically, it's not very useful for anything in common usage.  What
would you want it for ?...  sure it's nice information, but in practice,
unused dependencies from former ports get gc'd with pkg_delete -a...

... and if you're actually tinkering with dependencies, it's generally
related to the ports tree proper, and there is a plethora of targets in
there... along with indeed, sqlports which can give you all the info that
you need, assuming a bit of sql magic.

Once you understand, show-reverse-deps, it's fairly easy to write
customized stuff that does similar things.

BTW, any supplementary tool that does similar things directly in shell
has exactly zero chance to be included in the distribution.

The offical parser for PackingList *is* the perl code, dependency handling
as well.

Anything else is very likely to miss special cases and break atrociously.



Re: why the c99 mandate?

2020-04-10 Thread Marc Espie
On Fri, Apr 10, 2020 at 11:55:00AM +, Mayuresh Kathe wrote:
> i am not a c hotshot, so pardon my ignorance.
> i read that all new code under openbsd has to be c99.
> may i know what's so special about c99 over c89 which has been under heavy
> use for so long?

Like duh, ISO-C99 bis mostly improvements on ISO-C90.  And it's fully 
supported now.  Why not use it ?

Pre-C90 C has also been under heavy use, but most people were very happy
to get rid of it.

I would recommend reading the C standards, looking at what changed between
K/C90/C99, and making your own *informed* opinion.

As for C99, huge improvements in built-in types immediately comes to mind.

Smart initializers as well...



Re: Ports: how to install dependencies from binaries?

2020-04-09 Thread Marc Espie
On Tue, Apr 07, 2020 at 11:29:50PM -0400, Daniel Jakots wrote:
> On Wed, 8 Apr 2020 13:12:54 +1000, Stuart Longland
>  wrote:
> 
> > Silly question… how do you install the dependencies of a port from
> > binaries automatically?
> 
> https://man.openbsd.org/bsd.port.mk#FETCH_PACKAGES but it doesn't work
> very reliably, sadly.

Even after the recent fixes ?

Works just fine as long as your ports tree is in-synch with the repositories.



Re: Start point to learn OpenBSD programming

2020-03-16 Thread Marc Espie
On Mon, Mar 16, 2020 at 10:00:31PM +0100, Ingo Schwarze wrote:
> Hi Martijn,
> 
> Martijn van Duren wrote on Mon, Mar 16, 2020 at 09:24:26PM +0100:
> > On 3/16/20 9:22 AM, Ingo Schwarze wrote:
> >> Martijn van Duren wrote on Mon, Mar 16, 2020 at 08:52:54AM +0100:
> 
> >>> On 3/16/20 8:23 AM, Martin wrote:
> >>> If you want reading material find a function you don't understand and
> >>> lookup the manpage. If you want to have a more adventurous approach:
> >>> $ PAGE=$(ls /usr/share/man/man[23] | sort -R  | head -1); \
> >>> man ${PAGE##*.} ${PAGE%.*}
> 
> >> That can be simplified:
> >>   $ man -l $(ls /usr/share/man/man[23]/*.[23] | sort -R  | head -1)
> 

If you install random_run from packages,
you can go for

rr -1 man -l -- /usr/share/man/man[23]/*.[23]

that's precisely the kind of stupid thing it was built to handle.



Re: man to render pure text? (or a pipe in vi macros ?)

2020-03-04 Thread Marc Espie
On Wed, Mar 04, 2020 at 03:42:47PM +0100, Marc Espie wrote:
> On Mon, Mar 02, 2020 at 06:25:47PM +0100, Ingo Schwarze wrote:
> > Yikes.  I had no idea what either of these are doing and had to
> > try them out.  vi(1) contains so much bloat that is never really
> > needed and doesn't belong in a text editor at all.
> 
> No, all of this does belong in a text editor.
> 
> I use vi/vim with escapes *all the time*.
> 
> writing some text, having to do some computation,
> 
> type 42+15, select it !bc, and that's it, instant computations.
> 
> 
> seamless integration between shell and text editors is something that's
> in both major editors on Unix (vi family and emacs family).
> 
> If you're not using that, you are in the minority compared to power users.

Heck, piping to sort, or wc + undo   are two of the most common used 
commands.  Under vi, !}fmt  is also a favorite, though vim does have
better integrated commands...



Re: man to render pure text? (or a pipe in vi macros ?)

2020-03-04 Thread Marc Espie
On Mon, Mar 02, 2020 at 06:25:47PM +0100, Ingo Schwarze wrote:
> Yikes.  I had no idea what either of these are doing and had to
> try them out.  vi(1) contains so much bloat that is never really
> needed and doesn't belong in a text editor at all.

No, all of this does belong in a text editor.

I use vi/vim with escapes *all the time*.

writing some text, having to do some computation,

type 42+15, select it !bc, and that's it, instant computations.


seamless integration between shell and text editors is something that's
in both major editors on Unix (vi family and emacs family).

If you're not using that, you are in the minority compared to power users.



Re: size of size_t (diff angle)

2020-02-26 Thread Marc Espie
On Wed, Feb 26, 2020 at 11:01:56PM +0100, zeurk...@volny.cz wrote:
> Haai,
> 
> "Marc Espie"  wrote:
> > On Tue, Feb 25, 2020 at 08:56:06AM +0100, zeurk...@volny.cz wrote:
> >
> > You're looking at the wrong type. size_t is very good for what it does.
> 
> Yes; meproblem is with the 'what it does' part.
It represents memory sizes. It works on anything with a sane
memory model.

> > Try uintptr_t
> 
> Are you proposing a change to struct iovec?

Why should I ? readv works with sizes, so size_t is adequate.

You were mentionning caddr_t earlier. intptr_t and uintptr_t are
the adequate types for working with addresses. size_t is the adequate
family for working with sizes.

POSIX kind-of implies readv, which means that both realms tend of
mesh.

If you're on something where they don't, you're fucked.

Good luck.

What are you doing asking questions on an OpenBSD list, btw ?



Re: "not MAP_STACK" message in dmesg / system message buffer

2020-02-26 Thread Marc Espie
On Tue, Feb 25, 2020 at 08:30:11PM -0500, Andre Smagin wrote:
> Hello.
> 
> While prototyping something in C, I made a mistake with
> pre-processor macros, which I narrowed down to this:
> 
> int
> main()
> {
>   char *test[10][2097152] = { { 0 } };
> }
> 
> Running it results in
> $ ./a.out
> Segmentation fault (core dumped) 
> 
> and it also logs it in dmesg as
> 
> Feb 25 20:05:49 hamlet /bsd: [a.out]52048/372328 sp=7f7ff5fd4150 inside 
> 7f7fff7d5000-7f7d5000: not MAP_STACK
> Feb 25 20:06:49 hamlet /bsd: [a.out]94530/186499 sp=7f7ff5fe58c0 inside 
> 7f7fff7e7000-7f7e6000: not MAP_STACK
> Feb 25 20:07:09 hamlet /bsd: [a.out]9523/344960 sp=7f7ff5fd9fd0 inside 
> 7f7fff7db000-7f7db000: not MAP_STACK
> 
> I have not seen a segfaulting program being logged in system
> message buffer before. Is it expected behaviour?
> Just curious, the message was a bit confusing.

The stack has never been infinite.  Allocate too much, run into other issues.
ulimit -a  will already tell you about stack size limits.

Baring that, well, running into already allocated memory will hose you.
No further test necessary since MAP_STACK entered the OS (that's sickeningly
beautiful btw)



Re: size of size_t (diff angle)

2020-02-26 Thread Marc Espie
On Tue, Feb 25, 2020 at 08:56:06AM +0100, zeurk...@volny.cz wrote:
> Haai,
> 
> The definition of size_t keeps biting me.
> 
> Some background: in nnx, me's been using the equiv of caddr_t for
> counts. This works well; yet, while writing against existing code that
> uses size_t, an issue has surfaced.
> 
> First of all, let us reflect upon the definition of size_t in C99.
> 
> > size_t
> > which is the unsigned integer type of the result of the sizeof
> > operator;
> 
> That's not very specific. It kind-of implies that SIZE_MAX (defined
> later in the standard) is the largest possible offset, but not
> necessarily the largest possible address. This reeks of i86 real mode
> semantics, obsolete (for general-purpose machines) already when the
> PDP-11 was new.
> 
> POSIX is even less helpful:
> 
> > size_t
> > Used for sizes of objects.
> 
> (Let me note in passing that medisapproves of the significant overlap
> between C99 and POSIX, and the shameless disregard, in both, of the
> byte-oriented nature of UNIX and C).

You're looking at the wrong type. size_t is very good for what it does.

Try uintptr_t



Re: What TERM fixes Emacs?

2020-02-26 Thread Marc Espie
On Mon, Feb 24, 2020 at 09:35:21PM -0800, Emilia wrote:
>  
> 
> It is impossible to use Emacs on OpenBSD Terminal (no X). 
> 
> Look at this screenshots: 
> 
> On Linux / macOs -- this same version of Emacs and org-mode would
> display this file with colors etc. 

As stuart said, pccon is the best fit.

For a more elaborate answer: choosing a default TERM value for the console
is complicated.   The main issue being that a lot of the time, you will
ssh into another machine that probably doesn't run the same operating system,
and thus doesn't necessarily have the same termcap database.

So, the default choice in OpenBSD is to have a default value that's very
safe (vt220 have been around forever) so that you can login about anywhere.

... whereas pccon is way more recent and specific, and thus will probably
hose you when you try to login on some old machines.

(these days, new OS versions will all use the same termcap source, so you're
probably safe on anything released over the past 5 years)



Re: texlive_texmf-full package broken on cdn.openbsd.org

2020-01-29 Thread Marc Espie
On Tue, Jan 28, 2020 at 05:05:51PM -0600, Manuel Solis wrote:
> Hello Misc,
> 
> Just to confirm that the package textlive_texmf-full is broken on
> cdn.openbsd and cloudfare.cdn.openbsd
> 
> Error:
> doas pkg_add textlive_texmf-full
> quirks-3.182 signed on 2020-01-25T17:59Z
> https://cloudfare.cdn.openbsd.org/pub/OpenBSD/6.6/packages/amd64/textlive_texmf-full-2018.tgz:
> ftp: SSL write error: handshake failed: unexpected EOF
> signify: gzheader truncated
> Couldn't install texlive_texmf-full-2018
> 
> Using release 6.6 with all the patches thanks to syspatch.

Try with http instead of https ?

Packages are independently signed anyway, so it doesn't give you anything
in term of package install security.



Re: Can't locate OpenBSD/Quirks.pm in @INC

2020-01-19 Thread Marc Espie
On Sat, Jan 18, 2020 at 01:41:20PM +0100, Antoine Jacoutot wrote:
> On Fri, Jan 17, 2020 at 07:46:23PM -0700, myml...@gmx.com wrote:
> > 
> > On 1/17/20 7:25 PM, Jordan Geoghegan wrote:
> > > 
> > > 
> > > On 2020-01-17 18:10, myml...@gmx.com wrote:
> > > > HI,
> > > > 
> > > > 
> > > > I downloaded the install66.fs snapshot today, 20200117, and did a fresh
> > > > install.  Even though I got the full install set, i used http from
> > > > ftp.openbsd.org as the install source.
> > > > 
> > > > Installation went fine but when I tried to install packages I get the
> > > > above error.
> > > > 
> > > > "# pkg_add -vn pftop
> > > > quirks-3.216 signed on 2020-01-17T19:15:00Z
> > > > quirks-3.216: ok
> > > > Can't load quirk: Can't locate OpenBSD/Quirks.pm in @INC (you may need
> > > > to install the OpenBSD::Quirks module) (@INC contains:
> > > > /usr/local/libdata/perl5/site_perl/amd64-openbsd
> > > > /usr/local/libdata/perl5/site_perl /usr/libdata/perl5/amd64-openbsd
> > > > /usr/libdata/perl5) at /usr/libdata/perl5/OpenBSD/AddDelete.pm line 350.
> > > > 
> > > > pftop-0.7p19: ok
> > > > Merging manpages in /usr/local/man: /usr/local/man/man8/pftop.8
> > > > Extracted 252817 from 253475"
> > > > 
> > > [snip]
> > > 
> > > I believe quirks gets automatically installed when you install your
> > > first package.
> > 
> > 
> > AH HA, that seems to be the case.
> > 
> >  pkg_add -v pftop
> > quirks-3.216 signed on 2020-01-17T19:15:00Z
> > quirks-3.216: ok
> > pftop-0.7p19: ok
> > Extracted 252817 from 253475
> > 
> > 
> > I was just initially trying to see what would be installed without
> > actually installing.  I've run into issues before where the base system
> > packages and the userland stuff, if they aren't labeled the same date
> > have library issues.  I was trying to make sure i'd avoid that.
> > 
> > 
> > Thanks for the quick answer!
>  
> That is still a pkg_add bug though...

Thinking some more about it, I'm not sure about the right solution.
- I can easily suppress the quirks warning if -n is given.

- but then, quirks is kind-of part of pkg_add proper, so maybe it would be
more appropriate to install quirks anyway.   This is a bit unexpected, though.



Re: Can't locate OpenBSD/Quirks.pm in @INC

2020-01-18 Thread Marc Espie
On Sat, Jan 18, 2020 at 01:41:20PM +0100, Antoine Jacoutot wrote:
> On Fri, Jan 17, 2020 at 07:46:23PM -0700, myml...@gmx.com wrote:
> > 
> > On 1/17/20 7:25 PM, Jordan Geoghegan wrote:
> > > 
> > > 
> > > On 2020-01-17 18:10, myml...@gmx.com wrote:
> > > > HI,
> > > > 
> > > > 
> > > > I downloaded the install66.fs snapshot today, 20200117, and did a fresh
> > > > install.  Even though I got the full install set, i used http from
> > > > ftp.openbsd.org as the install source.
> > > > 
> > > > Installation went fine but when I tried to install packages I get the
> > > > above error.
> > > > 
> > > > "# pkg_add -vn pftop
> > > > quirks-3.216 signed on 2020-01-17T19:15:00Z
> > > > quirks-3.216: ok
> > > > Can't load quirk: Can't locate OpenBSD/Quirks.pm in @INC (you may need
> > > > to install the OpenBSD::Quirks module) (@INC contains:
> > > > /usr/local/libdata/perl5/site_perl/amd64-openbsd
> > > > /usr/local/libdata/perl5/site_perl /usr/libdata/perl5/amd64-openbsd
> > > > /usr/libdata/perl5) at /usr/libdata/perl5/OpenBSD/AddDelete.pm line 350.
> > > > 
> > > > pftop-0.7p19: ok
> > > > Merging manpages in /usr/local/man: /usr/local/man/man8/pftop.8
> > > > Extracted 252817 from 253475"
> > > > 
> > > [snip]
> > > 
> > > I believe quirks gets automatically installed when you install your
> > > first package.
> > 
> > 
> > AH HA, that seems to be the case.
> > 
> >  pkg_add -v pftop
> > quirks-3.216 signed on 2020-01-17T19:15:00Z
> > quirks-3.216: ok
> > pftop-0.7p19: ok
> > Extracted 252817 from 253475
> > 
> > 
> > I was just initially trying to see what would be installed without
> > actually installing.  I've run into issues before where the base system
> > packages and the userland stuff, if they aren't labeled the same date
> > have library issues.  I was trying to make sure i'd avoid that.
> > 
> > 
> > Thanks for the quick answer!
>  
> That is still a pkg_add bug though...

Known bug, it's on my list (need a clean machine to reproduce, because
I most often have so many packages installed it's not even funny)



Re: SSIZE_MAX

2020-01-16 Thread Marc Espie
On Thu, Jan 16, 2020 at 09:35:38AM +, cho...@jtan.com wrote:
> Raymond, David writes:
> > I am confused about SSIZE_MAX and read(2)/write(2).  The POSIX
> > SSIZE_MAX is something like 2^15 -1.  This seems to be a real
> > limitation when writing to a TCP/IP socket, as I learned from
> > experience.  However, much larger reads and writes seem to be possible
> > to files and UNIX sockets (pipes).  This makes me uneasy, given the
> > warning in the man pages for read(2)/write(2).
> >
> > Any insight on this topic would be appreciated.
> 
> I would guess this is part of the reason why ssize_t was invented
> - so that half of the numeric range could be wasted in order for a
> function to be able to return -1, and/or ridiculous notions of
> symmetry.

Most people out there don't understand C numeric types, as a very
long list of signed vs unsigned bugs in all kinds of tools show.

Clamping down to a value that has less chance of confusing the
numerous idiots out there seems like a valid approach to me.



Re: Awaiting a diff [was: Re: File systems...]

2020-01-10 Thread Marc Espie
On Fri, Jan 10, 2020 at 11:28:07AM +0100, Stefan Sperling wrote:
> On Fri, Jan 10, 2020 at 12:52:44PM +0300, Consus wrote:
> > On 20:06 Thu 09 Jan, Marc Espie wrote:
> > > It's been that way for ages. But no-one volunteered
> > > to work on this.
> > 
> > Anyone even knows about this? Aside from OpenBSD developers (who have
> > their plates full already) how an average person can find out that there
> > is rusty piece of code that should be taken care of?
> 
> Don't start looking at other people's problems before you can help yourself.
> Rewriting automounter for Theo as a first contribution is not likely to work.
> Find something small that annoys *you* and get involved by fixing that first.
> Learn how to approach and debug issues that affect you directly.


Ditto.

There are lots and lots of pieces in OpenBSD that could use some "obvious"
improvements, and not enough developers to do it all these days.

Ask anyone with an OpenBSD account how he got it, you'll get the same
story: use the OS, get annoyed by a small thing, start sending patches.

Very soon, you get to be responsible for some stuff.

No magic, just use the OS and scratch the itches.



Re: Awaiting a diff [was: Re: File systems...]

2020-01-09 Thread Marc Espie
If you want a useful project related to filesystems,
try the automounter.

Yes, that ancient code.

Look very closely. It has tendrils in NFSv2.

And some people, most prominently Theo, use amd(8).

Write an automounter that does not depend on NFSv2,
and then, most probably we can kill NFSv2.

Small steps...

It's been that way for ages. But no-one volunteered
to work on this.



Re: Awaiting a diff [was: Re: File systems...]

2020-01-09 Thread Marc Espie
On Thu, Jan 09, 2020 at 09:07:38AM +1000, Stuart Longland wrote:
> On 9/1/20 12:56 am, Ian Darwin wrote:
> >> - If we could clean-room implement a BSD-licensed
> >> EXT3/EXT4/BTRFS/XFS/JFS/whatever, following style(8), would there be
> >> interest in supporting that in OpenBSD?
> > 
> > And which "we" are you referring to here? Did you mean yourself,
> > or are you hoping that "somebody" will do it?
> 
> I'm hoping it will be more than one person assisting in this, and yes, I
> include myself in that group.

One useful thing you could do is review the diffs that are routinely
sent to various OpenBSD mailing-lists.

For instance, I have sent a few make-related diffs recently, on
which I'm still awaiting reviews.

How is this related to filesystem work, you may ask ?

Well, it's very related, though indirectly.

See, there aren't that many people actually doing the work
in OpenBSD.  A lot of it is routine work, but vital for
the project.

Queue Ingo and mandoc.
Queue me and make... and various other things.

and lots of other developers at time. Better lld support.
continuing the work started on ctf, etc, etc.


If there were more people actually *helping* instead of talking shit,
that stuff would go forward, and maybe, just maybe, us and other
developers *could* allocate time to do other stuff.

That might (or might not) include filesystem work.

Having the whole infrastructure working better would definitely help.

But no, it's way more fun to just sit there and say "hey, I only want
to do the sexy stuff. Please pre-chew it for me to baby-vomit stage so
I can be smug about it".



Re: But there is Fossil...

2020-01-06 Thread Marc Espie
On Mon, Jan 06, 2020 at 09:34:55PM +0100, Anders Andersson wrote:
> One good thing with this trainwreck of a discussion is that it pointed
> me to GoT. I've been looking for an alternative to CVS on my Amiga,
> but git is too convoluted to even start trying to build on a
> mostly-C89-semi-POSIX system. GoT seems like a much nicer starting
> point.
> 
> 
Good luck with that. I'm not quite sure Matt Dillon's unix compatibility
goo is going to be enough to convince amigaos to build got



Re: Automated OS builds?

2020-01-05 Thread Marc Espie
On Sun, Jan 05, 2020 at 06:08:55PM +, Paul Suh wrote:
> On Jan 5, 2020, at 12:43 PM, Morten Gade Liebach  wrote:
> > 
> > Read release(8), then write a script runs through the described process.
> 
> I can do that, and will if I have to, but if someone has already done it or 
> has a base to start from that would be better. (I’ve been building OpenBSD 
> releases that way since 3.2? 3.3? Something like that.) 

There are so many specifics to how each person configures his system and 
curates his local changes,
it's hard to give a "one size fits all".

(the complementary page for ports is bulk(8) btw
which goes to great length explaining the problems
you might encounter due to the size of the ports tree)



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Thu, Jan 02, 2020 at 11:52:03PM +0100, Marc Chantreux wrote:
> > You have something like 3 lines of perl to play with ;)
> 
> is there a todo list somewhere ?

More or less in my head, with lots of subprojects progressing at any given
time.

- I want to retire PackageLocator and have more correct packagerepository
lists... Update.pm is somewhat hackish;
- the virtual file system (Vstat.pm) is too simple and somewhat broken;
- there are still a few bugs in dependency handling;
- pkg_info should probably be cleaned up at some point
- there is some complicated work to speed up pkg addition by going through
a kind of "proxy", exactly like pkg_add-over-ssh works... this part is not
perl, though.
- pkg_create handling of dependencies completely misses @tag currently
- lib-depends-check is a complete mess and doesn't work with subdirectories

- the tests in regress/usr.sbin/pkg_add are woefully inadequate.

- dpb doesn't support running tests, and it's intended to take on portroach
capabilities at some point.
- it should have a "disconnected mode" with just ssh and no nfs. Quite possible
now that we have rsync in base.
- I'd like to integrate proot a bit more... the way proot is engineered to
prefer hardlinks over copy  was intended to make it possible to "quickly"
create a separate chroot for each build (it's somewhat linked to the previous
point, as both require precise accounting of packages).

there are more, but those are the ones coming up at the top of my head.



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Fri, Jan 03, 2020 at 09:43:21AM +1000, Stuart Longland wrote:
> On 3/1/20 8:50 am, Marc Chantreux wrote:
> >> Like this thread, or worse?
> > * long doesn't mean endless
> > * sharing points of view is never sterile (yours is inspired by other
> >   ones, right?)
> 
> I would say it's been highly educational.
> 
> Granted, this did not get off to a good start with the "let's replace
> Perl with Lua" debate, but it has piqued my interest in what the Raku
> team are up to.
> 
> It's pointed out style(9) which I'm having a read of now.  Having gotten
> familiar with the Linux kernel coding style and the coding style used in
> OpenThread, it's helpful sometimes to look at how others do it, as
> sometimes you can learn something that ultimately makes your life easier.
> 
> There's a valid point about whether this is the appropriate forum for
> this.  Question is, if not here, then where?

Any modern mailreader can easily tag messages as thread, so it's trivial to
avoid a given thread, as long as people don't fuck around with the
In-Reply-To info.



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Thu, Jan 02, 2020 at 04:10:43PM -0500, Paul Wisehart wrote:
> On Thu, Jan 02, 2020 at 09:12:42PM +0100, Marc Espie wrote:
> > 
> > Here are my current guidelines for OpenBSD perl tools.
> > 
> 
> Can you eleborate in greater detail?
> 

Not really, just go read the code and ask questions.

You have something like 3 lines of perl to play with ;)



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Thu, Jan 02, 2020 at 02:40:25PM -0600, danieljb...@icloud.com wrote:
> What if you want named parameters? (i.e. sending a hash as your
> argument)
> 
> sub m4
> {
> my $self = shift;
> my %args = @_;
> 
> # and then optionally
> my ($arg1, $arg2, $arg3) = @args{qw/arg1 arg2 arg3/};
> 
> # or you can just use $args{arg1}, etc...
> }

Such cases are a refactoring waiting to happen. If your parameters
get complicated enough that you want to name them, these's usually an
object hiding in there :)



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Thu, Jan 02, 2020 at 03:24:41PM -0500, Chris Bennett wrote:
> mod_perl, from reading the mailing list, looks like it will die off
> before long. Lack of developers and funding and interest given all the
> newer replacements.

Don't even think about using mod_perl these days.

Fast-cgi is the way to go. Even if you use something else but Dancer,
I'd urge you to read the documentation, it has a whole fucking manpage
about Dancer::Deployment



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Thu, Jan 02, 2020 at 07:49:28PM +0100, Marc Chantreux wrote:
> On Thu, Jan 02, 2020 at 10:42:54AM -0600, danieljb...@icloud.com wrote:
> > I don't understand why people say that perl's flexibility is a negative.
> 
> because sometimes, flexibility permit some endless sterile debates about
> the coding style.

Well, OpenBSD has got style(9). I have some specific adaptations for perl,
because a lot of the rules are for C.


Here are my current guidelines for OpenBSD perl tools.

In general, things are written following style(9) adapted for perl.

Specifically,
- named sub *are* functions.

So

sub f
{
my ($arg1, $arg2) = @_;

... code

}

- three styles of parameter grab for methods:


sub m1
{
my $self = shift;
}

No other parameter.

sub m2
{
my ($self, $p1, $p2) = @_;
}

when getting all parameters (no check on the number usually)


sub m3
{
my $self = shift;
...
do_something_with(@_);
}

for functions with unlimited parameters after the first one

(dubious whether this changes anything for performance reasons)

- wantarray should *only* be used for optimization purposes (yes/no answer
instead of full list).   Doing otherwise utterly complicates matters.


- I almost never put extra parentheses, and use the "4 space indent" rule
for continuing statements.

- chained index lookups should ditch the extra ->  .
prefer $self->{a}{b}  to $self->{a}->{b}

- don't put quotes around indices unless absolutely necessary (keywords)
and don't use keywords for keys.


- anonymous subs are part of the code:
So:
my $s = sub {
my $self = shift;
...
};


Note a full indent because the inside looks like code.

- modern perl prefered, so
$value //= something;
prefered over
$value = something  if !defined $value;

- autovivification welcome.

push @{$self->{list}}, value;
is perfectly fine without defining $self->{list} first.
Note that if (@{$self->{list} > 0)  *won't* autovivify list, so it can be
used for "does the list exist and is not empty" instead of 
if (exists $self->{list}


- I should probably normalize towards banning implicit return ?

- should I prefer "always refs"  over explicit % / @ ?
There is a slight legibility problem:
my @l;  is more readable than
my $l;  (this is a list)
and
my $l = [];   takes slightly more memory.


- most things unless explicitly being debugged should set
$DB::inhibit_exit = 1  right afer a fork and before an exec.


And I have some further general rules, learnt from past mistakes.

The perl package tools follow some stylistic and practical guidelines
- all new development should be object-oriented.
Have a package under either OpenBSD or DPB, and pass operations
to a constructed object (generally name the constructor new unless
you have better options) if you need to keep state, or to the
class name proper.

Examples:

my $pkgpath = DPB::PkgPath->new('devel/quirks'):

say "Normalized version is ",  $pkgpath->fullpkgpath;

$state->errsay(OpenBSD::Temp->last_error);

older code sometimes does not respect that.
It hasn't been converted because it's currently not worth it.
But there have been many instances where I've actually regretted
not doing things that way sooner.

The object itself is usually called "$self" unless there are reasons
not to.

Since there are no access control restrictions in perl, most often
internal methods are just prefixed with _.

Stylistically, methods without parameters don't need parameters, so
I don't write them, prefer $object->foo  to $object->foo()

It makes it less cumbersome to chain methods, e.g., $object->foo->bar(whatever);

- in the interest of chaining methods, stuff that tweaks an object should
return the object itself, so that

$self->set_foo(1)->set_bar(2)->run

will actually work

- a lot of code creates "unique" objects.
The pattern is to have a %cache hash in the package, and have the normal
constructor do things under the radar, calling create as needed.
create won't normally be used by client code.

- a lot of code creates "just in time objects".
Error.pm containt the OpenBSD::Auto class, that can be used to create
jit values, it contains one single construct, cache, that is used like so:
OpenBSD::Auto::cache(solver,
sub {
require OpenBSD::Dependencies;
return OpenBSD::Dependencies->new(shift);
});

so that the first call to $self->solver(x)
will instantiate $self->{solver} to the required object.
And that call and all subsequent calls will return the same object.


- there are way less files than classes. Things are organized in a
"put a whole set of related things together in the same file".
Full OO also means you don't need to use Foo; from the start, you can 
require Foo; dynamically, thus loading it later.  This does speed up the
startup of tools significantly.

- in general, singletons are frowned upon. We still have a few (list ?),
mainly as cached values in specific packages.  There 

Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Thu, Jan 02, 2020 at 04:22:08PM +0100, Marc Chantreux wrote:
> hello,
> 
> > > my %user = qw(
> > > login  mc
> > > shell  /bin/zsh
> > > );
> > > print $user{login};
> 
> > my %user = ( login => 'mc', shell => 'bin/zsh');
> > is way more readable in that case, I think,
> > and it does showcase what a *smart* quoting system can do.
> 
> well ... i prefer the way i wrote because i love to:
> 
> * remove useless symbols
> * read columns

Well, => and ,   allow to figure out errors in odd/even hashes easily.

I will always lean towards idiot-proofing the code.



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Thu, Jan 02, 2020 at 12:40:51PM +0100, Marc Chantreux wrote:
> the quoting system
> 
> # qw( for a list of barewords )
> my %user = qw(
> login  mc
> shell  /bin/zsh
> );
> print $user{login};

I wouldn't write it that way

my %user = ( login => 'mc', shell => 'bin/zsh');

is way more readable in that case, I think,
and it does showcase what a *smart* quoting system can do.



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-02 Thread Marc Espie
On Thu, Jan 02, 2020 at 07:34:22PM +1000, Stuart Longland wrote:
> On 2/1/20 12:30 am, Marc Chantreux wrote:
> > * the python community was unfair comparing the langages (using ugly
> >   perl code and nice python counterparts). instead of taking time to
> >   explain all the biases, perl community repetedly asserted that the
> >   authors of those article were incompetents and gone away.
> 
> Heh, I've heard Perl described as executable line noise, and for sure,
> it will let you write code like that.
> 
> But so does C.  There's even a contest for doing exactly that.
> 
> I've seen some pretty ugly Python code too.

Not to beat a dead horse, but most of the python configury stuff,
including scons, is pretty shitty.   Lots of really bad pseudo-OO stuf
(hey let's use that cool feature just because we can)

I hate when I have to fix python configure... it looks like a 
bunch of complete beginners set up to reinvent a square wheel.

python is definitely my #1 most-hated language when fixing configure in
ports. Yes, it beats autoconf and libtool by a large margin.



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-01 Thread Marc Espie
On Wed, Jan 01, 2020 at 04:44:48PM +0100, Marc Chantreux wrote:
> > I still thing DBIx::Class is overkill. The DB::Rose stuff was way simpler
> > and I would have preferred for it to win.
> 
> Well... i liked the simplicity until i had some cases like having 2
> different DBs with the same model: piece of cake with DBIxC and
> impossible with ActiveRecord (AFAIR).

Oh, I'm not talking ActiveRecord.

Did you ever look at the suite of modules from John Syracusa (DB::Rose and
the like) ? fairly clean and nice.



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-01 Thread Marc Espie
On Wed, Jan 01, 2020 at 03:43:38PM +0100, Marc Chantreux wrote:
> hello,
> 
> > The only thing that's really missing in perl is proper thread support.
> > Don't know if that's going to happen.
> 
> seems ... complicated ...

Tell me about it. The only existing thread support  was so clunky it got
thoroughly deprecated.   It was really bad back in userland pthread days,
because you couldn't even build perl binaries depending on threaded libraries
(all-or-nothing -pthread flag) such as frozenbubble.

> > I have a wish-list of things that are not that likely to happen, I would
> > like to be able to use prototypes on methods, for instance.
> 
> what do you mean by this? prototypes are here for decades and signatures
> are experimental and i guess it will be core in some releases.


You can't mix oo lookup and prototypes.

Stuff like 
$o->method { code }
for instance.

you have to use the clunkier
$o->method(sub { code });

> > Perl also missed a turn for web development. I think Catalyst was a huge
> > mistake (hey, you've got *choices* everywhere. Let's confuse everyone),
> 
> perl had CGI.pm, maypole, mod_perl, catalyst, jifty, dancer, mojolicious ...
> Template toolkit is still by far the best template toolkit i know.
> i really thing the only thing where perl was not a precursor in web dev
> is plack (which is inspired by wsgi which is inspired by rack ... i
> don't know if there is another ancestor).

That's the big issue. Too much choice in the ecosystem, with some of it not
clearly enough explained... and no simple integration with js libraries for
ajax at first.

> you mean mason ? mason is the php of perl: don't organize your code:
> write a single page with everything in it ... it was a terrible thing
> to maintain (see the code of request tracker...).

Yeah, I mean mason.  At some point long ago, it was about the only 
game in town for perl.

> > Dancer was a few years too late to the party.
> 
> sinatra (from ruby) was the source of inspiration of Dancer which,
> AFAIK, appears years before flask and bottle.

> ActiveRecord was easier than DBIx::Class for simple situations. that's
> one of the reasons of the popularity of RoR (also the Ruby syntax).

I still thing DBIx::Class is overkill. The DB::Rose stuff was way simpler
and I would have preferred for it to win.



Re: Suggestion: Replace Perl with Lua in the OpenBSD Base System

2020-01-01 Thread Marc Espie
On Tue, Dec 31, 2019 at 11:56:46PM -0700, Bob Beck wrote:
> read fucking code.  change fucking things. send some fucking diffs. get
> fucking yelled at. learn from your fucking mistakes.  show some fucking
> passion.  filter fucking misc@ and all this useless bleating into the
> toilet.
> 
> none of us have time to spoon feed you in some “boot camp”
  ^fucking
> 
> there are two types of programmers. the self taught, and the hopeless. it
^fucking
> is your job to turn yourself from the hopeless to the self taught.
^fucking
> 
> shut up and fucking hack.
> 

There, you missed a few.



Re: Suggestion: Replace Perl with Lua in the OpenBSD Base System

2020-01-01 Thread Marc Espie
On Tue, Dec 31, 2019 at 09:06:38PM +0100, Christer Solskogen wrote:
> On Tue, Dec 31, 2019 at 5:50 PM Marc Espie  wrote:
> 
> > We did retire vax, and we no longer have any platform without dynamic
> > libraries.
> >
> >
> OT but: out of sheer curiosity, why didn't VAX support dynamic libraries?

Very different architecture, no register to do pic code efficiently and
512 bytes pages.

NetBSD moved to shared libraries on vax at a huge performance cost 
(something like 30%), if I recall what Miod was saying at the time.



Re: Suggestion: Replace Perl with Lua in the OpenBSD Base System

2020-01-01 Thread Marc Espie
On Wed, Jan 01, 2020 at 10:06:47AM +0100, Anders Andersson wrote:
> On Wed, Jan 1, 2020 at 4:51 AM Stuart Longland
>  wrote:
> 
> > Perl 6 will be a major change though, more disruptive than the Python2→3
> > mess was.  So we may be in for some "fun" in the near future.
> 
> Gotta stop this before it derails: perl 6 is not the next version of
> perl 5. It's not compatible, it's not an upgrade, it's a completely
> new language and does no longer even share the same name (renamed to
> raku). There is no "perl 6" that will replace perl 5.

Actually all the cool and useful ideas that perl6 had DID trickle down
into perl5 a few years ago.

Perl6 was (I think) intended as a test bed for ideas by Larry.  Everybody
got sidelined when a perl6 implementation came out of nowhere,
written by Audrey Tang, an extra-terrestrial years ahead of everyone.



Re: Suggestion: Replace Perl with Lua in the OpenBSD Base System

2020-01-01 Thread Marc Espie
On Tue, Dec 31, 2019 at 10:01:50PM -0500, Steve Litt wrote:
> On Tue, 31 Dec 2019 15:57:47 -0600
> Eric Zylstra  wrote:
> 
> > Proposing such a huge project without the ability to do it?  I may
> > have been a little disrespectful, but not the first one in the
> > thread.  And my point wasn’t to be disrespectful, but to point out
> > that most proposals unaccompanied by code and that don’t solve
> > obvious problems don’t seem to be received very well.  Apologies if
> > that wasn’t within bounds.
> 
> What if the OP had instead of the suggestion submitted two or three Lua
> scripts to replace two or three Perl scripts? Would you still have the
> same opinion? 

Good luck with that.

Tools in base written in perl:
- libtool
- pkg-config
- pkg_add

The libtool part is insane. pkg-config is doable.

You won't be able to rewrite pkg_add without rewriting the whole
ecosystem, because it's heavily based on a few choice modules (see
OpenBSD::Intro(3) )  and you more or less have to rewrite all of it
together, meaning:
-> pkg_add, create, delete
-> fw_update
-> dpb
-> update-plist
-> check-lib-depends
-> pkg_check-*
-> pkg_outdated
-> pkg_subst
-> port-resolve-lib-helper
-> proot
-> register-plist

There are a few other scripts which are independent of that framework,
but still that's some complicated job.


Instead of rewriting it, I would be way more interested in somebody
looking carefully at (say) libtool and fixing the missing parts...



Re: perl popularity inside openbsd community? (Re: Suggestion: Replace Perl ...)

2020-01-01 Thread Marc Espie
On Tue, Dec 31, 2019 at 10:36:15PM +0100, Anders Andersson wrote:
> Of course its age is showing in some areas but in my experience, those
> things are actually still worked on, and have been fixed without major
> incompatibilities (python3 anyone?).

The only thing that's really missing in perl is proper thread support.
Don't know if that's going to happen.

Its garbage collector is also slightly peculiar...  I remember looking
really hard for a leak because a file handle in an anonymous sub wouldn't
be properly collected (the one from pkg_add's progressmeter, actually)


I have a wish-list of things that are not that likely to happen, I would
like to be able to use prototypes on methods, for instance.


> I remember a few years ago when I was briefly researching a
> replacement for perl for my personal projects and I tried out python3
> and ruby in parallel and ruby was definitely the winner there. I have
> absolutely no idea why python even gained the popularity it has, it
> felt like a random hack, especially compared to ruby. The only thing I
> really miss from python is "yield".

Yeah, native coroutine support without a hack would be a blast.
Even C++ is getting that for its next main revision.

The popularity of python is partly explained by them catering more to
teaching needs.   As far as I know, there is no equivalent of the python
notebooks.  Stuff like jupyter means you don't even have to install 
complicated arcane stuff to learn python. Cool for the young pups.

Perl also missed a turn for web development. I think Catalyst was a huge
mistake (hey, you've got *choices* everywhere. Let's confuse everyone),
so a lot of people didn't transition from Meson to another perl module, but
instead switched to ruby-on-rails or something like that.

Dancer was a few years too late to the party.



Re: Suggestion: Replace Perl with Lua in the OpenBSD Base System

2019-12-31 Thread Marc Espie
On Tue, Dec 31, 2019 at 10:45:34PM +1000, Stuart Longland wrote:
> On 31/12/19 3:54 pm, Marc Espie wrote:
> > Contrary to what some people might think, the tools in question won't be
> > easier to understand and manage if written in another language.
> 
> I'm of the opinion that "if it ain't broken, don't fix it".  What is
> "broken" about Perl that we're trying to fix with a replacement (whether
> it be Lua, Python, NodeJS, Ruby, PHP, TCL, alb, BASIC … or something else)?

It used to be there was no choice at all.

Back then, perl was already in the base system and running on everything
*including vax*. Python (for instance) not so much. The standard build
definitely requires shared libraries.

We did retire vax, and we no longer have any platform without dynamic
libraries.

As for the license, python's license appears fairly similar to Perl's
artistic license.  I would worry a bit about the strong terms in

6. This License Agreement will automatically terminate upon a material breach of
   its terms and conditions.

for which no equivalent is visible in Perl's license.


You got to remember though that OpenBSD initially had a lot of stuff that
was not BSD licensed (the full toolchain!), but used for building (we got rid
of the fp emulation code fairly early on). Though that has changed on some
platforms (hurray for clang pre-apache license), I suspect there might be
some reticence to importing more stuff that's not strictly under a BSD/MIT
license.



Re: Suggestion: Replace Perl with Lua in the OpenBSD Base System

2019-12-31 Thread Marc Espie
On Tue, Dec 31, 2019 at 06:57:02AM -0600, Daniel Boyd wrote:
> As one of the few remaining people out there who considers perl to be their 
> favorite language—starting to wonder if it’s just me and Larry Wall at this 
> point—I’d like to say that perl should stay in base on its merits, all the 
> perl-based system tools notwithstanding.

You're definitely not alone, though I also enjoy other languages where perl
falters (long startup time and complexity of linking with libraries mean
I do C and C++ a lot as well)



Re: Suggestion: Replace Perl with Lua in the OpenBSD Base System

2019-12-30 Thread Marc Espie
Removing perl from base would be very painful.

I don't fancy rewriting all the perl tools in something else (specifically,
most of the ports and package infrastructure)

lua would definitely NOT be appropriate for that. The only half valid
candidate would be python.

Contrary to what some people might think, the tools in question won't be
easier to understand and manage if written in another language.



  1   2   3   4   5   6   7   8   9   10   >