Re: Allow install from https server w/ self signed cert

[Jacob L. Leifman]

On 7 Jan 2017 at 15:28, Theo de Raadt wrote:

> > On Fri, Jan 06, 2017 at 10:48:37AM -0500, RD Thrush wrote:
> > > On 01/06/17 06:28, Stuart Henderson wrote:
> > > > Related to this (and particularly thinking about autoinstalls),
> > > > would it make sense to allow explicit protocols in the hostname?
> > > > 
> > > > some.host -> https with http fallback
> > > > http://some.host/ -> http only
> > > > https://some.host/ -> https only, no fallback
> > > 
> > > That would totally work for my install problem.
> > > 
> > > FWIW, instead of running a patched install.sub, "rm
> > > /etc/ssl/cert.pem" makes the install bypass the https attempt.
> > > 
> > 
> > Note, if you're upgrading or otherwise have a way to et a cert.pem
> > bundle onto there to *replace* the default, you could always drop the
> > signer for your private self-signed server into the cert.pem bundle,
> > at which point it would be accepted as trusted. 
> > 
> > of course if you're just installing you have an interesting chicken
> > and egg problem, unless you put it somewhere on an https site that
> > does have a real certificate, drop out of the installer and do
> > 
> > ftp -o /tmp/mysigner.pem https://my.secure.site/mysigner.pem
> > cat /tmp/mysigner.pem >> /etc/ssl/cert.pem
> > 
> > then continue the install, and you're good. 
> > 
> > Almost wonder if it's worth an extra question in the installer to ask
> > for an https address to retrieve a certficiate bundle to be appended
> > to cert.pem for the install...
> 
> And we should also ask a firmware question?
> 
> Nope.  I don't think we should bend over backwards for people doing
> strange things.  They are on their own.
> 

Most of the time I agree with this particular attitude and it is indeed 
appropriate for the OP case. However, there some major networks such as 
various governments (or for example .mil) that do not participate in 
the public PKI but run their own PKI infrastructure. What effect will 
the installer's checks have in that environment? What workarounds would 
be reasonable and approriate? and does it make sense for OpenBSD to 
support such scenarios out-of-the-box to promote wider adoption of 
better software?

Re: rdist.c patch

[Jacob L. Leifman]

In some regards rdist is possibly more powerful/flexible than rsync 
with its built in support for post-processing scripts. For example, I 
find it extremely useful for keeping CARP firewalls in sync while 
allowing for administrative changes to either.

On 7 Mar 2016 at 11:18, Michael Kennett wrote:

> I use rdist (triggered from /etc/daily), but the standard build does have
> limitations and can dump core in some circumstances (refer
> http://marc.info/?l=openbsd-tech=141551244520320=2).
> 
> My use of it is to maintain a shadow boot partition, and to copy selected
> files onto a separate server. Whilst it is less powerful than rsync it is
> nice (in my view) to have a tool like this in the standard distribution.
> 
> Cheers, Michael
> 
> 
> 
> On Mon, Mar 7, 2016 at 10:12 AM, Alexander Hall  wrote:
> 
> >
> > >On 03/06/16 00:12, Philip Guenther wrote:
> > >> Anyone still *using* rdist?
> >
> > I really try to use it to sync my ~ dotfiles but the configuration is as
> > intuitive as that of sudo.
> >
> > As much as I like having that kind of tool in base, maybe it isn't all
> > that useful anymore.
> >
> > /Alexander
> >
> > >>
> > >>
> > >> Philip Guenther
> > >>
> > >/etc/daily
> > >
> > >if configured to do so that is.
> >
> >
> 

Re: openssl's *strlcy

[Jacob L. Leifman]

I'm guessing that openssl was incorporated into OpenBSD base without 
prior sufficient audit by the OBSD devs because it was presumed to have 
better auditing / quality control upstream given its security critical 
nature and function. (A number of devs have commented in the past about 
the [lack of] code style, but I get the impression no-one expected the 
degree of *sloppiness* now being uncovered.)  So here's a question, are 
there any other chunks of code that have been imported en-mass from an 
upstream source that could/should use an audit? especially, something 
that some of us non-developers might be able to assist with?


On 18 Apr 2014 at 19:06, Bob Beck wrote:

 
 On Fri, Apr 18, 2014 at 05:19:15PM -0700, Claus Assmann wrote:
  Seems it is ok to use strlcat/strlcpy that way in some cases:
  $ cat src/usr.sbin/smtpd/*.c | egrep -c ' strlc(at|py)\('
  249
 
 If your only goal is ensuring you don't have a non-nul terminated
 string, sure, that's great. and the way we like to indicate
 that you thought about that in code is to explicitly put (void)
 in front of it. 
 
 so i.e.
 
 (void) strlcat(foo, bar, size)
 
 should mean you dont care if the value is truncated. and that's
 perfectly ok, because lots of times you dont. 
 
 The problem is what if bad things happen if the string is truncated?
 
 I started looking just a short time ago, and will probably call it a
 night, why? well. my wife is asking me to put the computer down, and
 frankly, I need to rewrite the *second* function I got to.  For your
 viewing pleaseure, please check out line 1124 of apps/ca.c There you
 see two strlcpys.. no problem right, but wait, we're building a path
 we're adding stuff on the end, if that / doesn't get there there are
 problems. oh look, then there's pointer arithmatic and some pretty
 crufty goo. 
 
 This sort of tells me they never heard of asprintf.
 
 The point is in the *second usage* I go to look at it, I can't just
 look at that code and say oh it's fine. 
 
 Of necessity, some crypto code will be a bit scary to look at.
 anything that has to do regular jobs, like building a pathname, in a
 security library, should be a thing of beauty, not something that
 makes you want to throw up in your mouth. (and don't say it's just
 an app. I've seen what people do with this app and what
 they feed it data from..)
 
 sure, s/strlcpy/strncpy/ is 'better' - but this library is
 security code, and it should be clean, and modern. that's the goal
 here. not halfassed fixes with find and sed. If this is our fork, 
 it will not be halfassed.
 
 -Bob
 
 
 

(Fwd) last(1) nit

[Jacob L. Leifman]

ping?  ALL feedback is welcome :-)

--- Forwarded message follows ---
From:   Jacob L. Leifman jac...@bitwise.net
To: tech@openbsd.org
Date sent:  Fri, 25 Jan 2013 18:34:55 -0500

The last(1) utility allows for an alternate data file to be specified 
with -f switch which is particularly useful for reviewing rotated 
versions of wtmp or forensic analysis.

What I found distracting (and made me briefly think it was not working) 
is that regardless this specification, at the end last(1) always prints 
wtmp begins ... because this name is hardcoded in the output.

Below is a quick-and-dirty patch to use the actual name of the data 
file. The dirty refers to the fact that it prints the full pathname 
(e.g. /var/log/wtmp) even in the most common use case when all defaults 
are used. Although this may be objectionable under the doctrine of 
least change, I also see a potential benefit that it will always 
indicate the source of the data in an unambiguous manner.

--- last.c.orig Thu Oct 29 00:33:55 2009
+++ last.c  Fri Jan 25 18:20:10 2013
@@ -407,7 +407,7 @@
asctime(gmtime(total))+11);
}
ct = ctime(buf[0].ut_time);
-   printf(\nwtmp begins %10.10s %*.*s %4.4s\n, ct, timesize, 
timesize,
+   printf(\n%s begins %10.10s %*.*s %4.4s\n, file, ct, timesize, 

timesize,
ct + 11, ct + 20);
 }


--- End of forwarded message ---


WPM$4FA7.PM$
Description: Mail message body

last(1) nit

[Jacob L. Leifman]

The last(1) utility allows for an alternate data file to be specified 
with -f switch which is particularly useful for reviewing rotated 
versions of wtmp or forensic analysis.

What I found distracting (and made me briefly think it was not working) 
is that regardless this specification, at the end last(1) always prints 
wtmp begins ... because this name is hardcoded in the output.

Below is a quick-and-dirty patch to use the actual name of the data 
file. The dirty refers to the fact that it prints the full pathname 
(e.g. /var/log/wtmp) even in the most common use case when all defaults 
are used. Although this may be objectionable under the doctrine of 
least change, I also see a potential benefit that it will always 
indicate the source of the data in an unambiguous manner.

--- last.c.orig Thu Oct 29 00:33:55 2009
+++ last.c  Fri Jan 25 18:20:10 2013
@@ -407,7 +407,7 @@
asctime(gmtime(total))+11);
}
ct = ctime(buf[0].ut_time);
-   printf(\nwtmp begins %10.10s %*.*s %4.4s\n, ct, timesize, 
timesize,
+   printf(\n%s begins %10.10s %*.*s %4.4s\n, file, ct, timesize, 
timesize,
ct + 11, ct + 20);
 }

Re: base httpd ProxyPreserveHost not working

[Jacob L. Leifman]

On 25 Mar 2012 at 10:22, Stuart Henderson wrote:

 On 2012/03/24 21:33, Stuart Henderson wrote:
  On 2012/03/24 16:58, Jacob L. Leifman wrote:
   following up on my own post  (the only reponse I received was the
   suggestion to switch to nginx, and while it does appear that much
   development is happening there, I am not in a position to deploy
   current right away...)
  
  Your analysis and diff look correct to me.
 
 as an offlist mail pointed out, this has a problem with the : in raw v6
 addresses.
 

I believe that most of that problem has already been mitigated in the 
evolution of the code and my corresponding adjustment of the patch. 
Specifically, after splitting the original destination string into 
hostname:destportstr, the two components are now used as strings 
without further mangling and are [almost] always recombined in a manner 
that ultimately results in the original string.

For the remaining spot where a raw v6 address might still get mangled 
(due to a component of the address matching the value of a standard 
port and being suppressed as a result), I offer the supplementary patch 
below:

--- proxy_http.c~   Sat Mar 24 14:29:30 2012
+++ proxy_http.cMon Mar 26 17:14:18 2012
@@ -389,7 +389,10 @@
AP_HOOK_DECLINE(DECLINED),
rc, r, f, hostname, destportstr, destportstr);
 if (rc == DECLINED) {
-   destportstrtonum = strtonum(destportstr, 0, 65535, errstr);
+   if (strchr(destportstr, ':') != NULL)
+   destportstrtonum = -1;  /* force output below */
+   else
+   destportstrtonum = strtonum(destportstr, 0, 65535, errstr);
if (errstr)
errx(1, The destination port is %s: %s, errstr, destportstr);
 

Since I am by no means an expert in IPv6, here is an alternative, very 
aggressive version of the supplementary patch:

--- proxy_http.c~   Sat Mar 24 14:29:30 2012
+++ proxy_http.cMon Mar 26 17:15:04 2012
@@ -389,7 +389,10 @@
AP_HOOK_DECLINE(DECLINED),
rc, r, f, hostname, destportstr, destportstr);
 if (rc == DECLINED) {
-   destportstrtonum = strtonum(destportstr, 0, 65535, errstr);
+   if (conf-preserve_host)
+   destportstrtonum = -1;  /* force output below */
+   else
+   destportstrtonum = strtonum(destportstr, 0, 65535, errstr);
if (errstr)
errx(1, The destination port is %s: %s, errstr, destportstr);
 

BTW, completely as an aside, can anyone demonstrate a potential use-
case scenario where ProxyPreserveHost On and raw v6 addresses would 
collide? Unless I am mistaken, the sole purpose of this directive is to 
pass the original requested FQDN to a NameVirtualHost backend.

Re: base httpd ProxyPreserveHost not working

[Jacob L. Leifman]

following up on my own post  (the only reponse I received was the 
suggestion to switch to nginx, and while it does appear that much 
development is happening there, I am not in a position to deploy 
current right away...)

On 23 Mar 2012 at 18:26, Jacob L. Leifman wrote:

 In the process of migrating a simple reverse proxy from a Linux box to
 OpenBSD (4.9-i386) I've come across a problem behavior that seems to
 indicate that the directive ProxyPreserveHost is not working. In fact
 setting it on or off does not affect the perceived behavior at all.
 
 The setup is rather simple and syntactically identical between the Linux
 Apache2 and OpenBSD base httpd (Apache 1.3+):
 
  VirtualHost [public_ip]
   ServerName  www.customer.com
   ErrorLoglogs/error_log
   CustomLog   logs/access_log common
   ProxyPreserveHost On
   ProxyPass   / http://10.xx.yy.zz/
   ProxyPassReverse/ http://10.xx.yy.zz/
  /VirtualHost
 
 The resulting behaviors differ thus:
 
 request http://www.customer.com/test/page.aspx --
 *want/linux: the backend server recognizes that the request is for
 virtual server named www.customer.com and delivers the required page;
 *openbsd: the backend server sees http://10.xx.yy.zz/test/page.aspx,
 does not recognize the site and returns 404 Page not found.
 
 request http://www.customer.com/ --
 *want/linux: the backend server recognizes the site and correctly forms
 all self-referential links on the home page; *openbsd: the backend
 server generates all self-referential links as http://10.xx.yy.zz/...
 which get handed back to the client outside the firewall, rendering the
 website not usable.
 
 I checked cvsweb for the proxy module of the base httpd, and it has had
 no commits in year, so I believe I am running the latest code.
 
 Please let me know if/what additional diagnostic info is needed, or
 hopefully patches to test.
 
 Thanks,
 -Jacob.
 
 

After some code archeology I discovered that a critical piece of the 
ProxyPreserveHost implementation which was committed to proxy_http.c 
r1.17 was lost because r1.18 evolved directly from r1.16.

I have reconstructed the missing patch and made a minor adjustment 
related to the evolution of code since then. Below is the resulting 
new patch, which does appear to fix my problem above. I would 
appreciate the community/devs reviewing it to make sure I did not miss 
something important.

--- proxy_http.c.orig   Mon Jan 10 15:05:49 2011
+++ proxy_http.cSat Mar 24 14:29:30 2012
@@ -151,7 +151,7 @@
 {
 const char *strp;
 char *strp2;
-const char *err, *desthost;
+const char *err, *desthost, *hostname;
 int i, j, sock,/* len,*/ backasswards;
 table *req_hdrs, *resp_hdrs;
 array_header *reqhdrs_arr;
@@ -362,21 +362,41 @@
 ap_hard_timeout(proxy send, r);
 ap_bvputs(f, r-method,  , proxyhost ? url : urlptr,  HTTP/1.1 
CRLF,
   NULL);
+
+if (conf-preserve_host) {
+   hostname = ap_table_get(r-headers_in, Host);
+   if (!hostname) {
+   hostname = r-server-server_hostname;
+   ap_log_rerror(APLOG_MARK, APLOG_WARNING, r,
+ proxy: No host line on incoming request 
+ and preserve host set forcing hostname to 
+ be %s for uri %s, hostname, r-uri);
+   }
+   strp2 = strchr(hostname, ':');
+   if (strp2 != NULL) {
+   *(strp2++) = '\0';
+   if (ap_isdigit(*strp2))
+   destportstr = strp2;
+   }
+}
+else
+   hostname = desthost;
+
 {
int rc = DECLINED;
ap_hook_use(ap::mod_proxy::http::handler::write_host_header, 
AP_HOOK_SIG6(int,ptr,ptr,ptr,ptr,ptr), 
AP_HOOK_DECLINE(DECLINED),
-   rc, r, f, desthost, destportstr, destportstr);
+   rc, r, f, hostname, destportstr, destportstr);
 if (rc == DECLINED) {
destportstrtonum = strtonum(destportstr, 0, 65535, errstr);
if (errstr)
errx(1, The destination port is %s: %s, errstr, destportstr);
 
if (destportstr != NULL  destportstrtonum != destport)
-   ap_bvputs(f, Host: , desthost, :, destportstr, CRLF, NULL);
+   ap_bvputs(f, Host: , hostname, :, destportstr, CRLF, NULL);
else
-   ap_bvputs(f, Host: , desthost, CRLF, NULL);
+   ap_bvputs(f, Host: , hostname, CRLF, NULL);
 }
 }

Re: [PATCH] fix broken support for installing on extended partitions

[Jacob L. Leifman]

I can confirm the following points:

 * the code in the kernel works correctly in that it can successfully 
mount and access deeply nested non-OpenBSD extended partitions;
 * the code in the base fdisk(8) utility works correctly as I have used 
it numerous times to fix disk partitioning on Windows and Linux boxes 
with sometimes over a dozen extended partitions;
 * in OBSD 4.4 and 4.5 I have done a test installation to every 
possible MBR and extended partition on a disk carved into about a dozen 
slices and was only successful with the MBR partitions and the very 
first two extended partitions;

 * the published documentation I was ever able to find describing the 
extended partitions in the IBM PC architecture is consistent with the 
implementation present in the installboot(8) utility, HOWEVER,
 * my examination of the actual data in the partition description 
sectors of numerous boxes partitioned with Windows or Linux has found 
it to be instead consistent with the description provided below by OP.


On 11 Feb 2011 at 14:48, Kenneth R Westerback wrote:

 On Fri, Feb 11, 2011 at 05:11:45PM +0200, ucs...@gmail.com wrote:
  On Fri, Feb 11, 2011 at 06:28:28AM -0500, Kenneth R Westerback
 wrote:
   On Fri, Feb 11, 2011 at 09:47:02AM +0200, ucs...@gmail.com
 wrote:
Despite the touted support, OpenBSD wouldn't be able to boot off
 an
extended partition, unless it's the very first or second one.

The kernel was fixed (after a fashion) some time ago, but
 boot(8) and
installboot(8) still live under the mistaken idea that extended
 partitions
are defined recursively, with each of them acting like a full
 MBR again
and again, each with 4 smaller partitions inside it, and the
 offset of
each subsequent extended partition should be calculated wrt. the
 offset
of the bigger one containing it.

That doesn't work that way and never worked that way: There is
 one big
extended partition defined in the MBR, and then inside it there
 is a
linked list of 'logical' partitions, each of them with a fake
 MBR
defining a data sub-partition (whose offset is from the start of
 this
logical partition), and possibly a link to the next logical
 partition
(whose offset is from the start of the main extended
 partition).

   
   Can you provide some pointers to where you are deriving this
   information and what testing you have done and what setups now
 work
   that did not before? You hint in your description, but this is
 an
   area that has been argued over many times with each fix breaking
   some systems while fixing others. The last test rig I set up had
  
  I don't see your point.
 
 My point was attempting to discover if you had discovered some clear
 docs on how this is supposed to work. Hope springs eternal. Also
 to discover what testing you had done to demonstrate that this fixes
 a problem without causing other side effects. If you had done such
 testing I would have to do less myself.
 
 Did you encounter a problem? What was it? Would other people care?
 How did you encounter it? Did you just read code and type up the
 diff (which looks good) on your Mac and hope it compiled/worked on
 OpenBSD?
 
  
  The kernel is already doing it the way I describe, though the code
  is a bit clunky and stupidly limited to 8 extended partitions.
  (that's the stuff in kern/subr_disk.c).
 
 Thanks for the moral support. The limit of 8 was one of those things
 that
 having a nice definitive doc would help with.
 
  
  Are you arguing for keeping boot(8) and installboot(8) doing a
 different
  thing from the kernel?
 
 No, I'm attempting to discover what work you had done that would not
 have
 to be repeated before considering committing. 
 
  
  It's easy to reproduce -- I could prepare you a qemu image with all
 things
  set up and the 3rd or 4th extended partition of type 0xa6, but you
 don't
  need such hand-holding. You can do that yourself ;-)
  
   Having working extended partitions would be nice, but finally
 finding
   definitive docs that clearly and correctly describe what is
 supposed
   to happen would be even better.
  
  There are no such docs. What I describe in the 2nd paragraph is the
 way
  windows (and linux, and probably others) work. 
 
 And you know windows works this way because  docs? testing?
 reverse compiling windows (which ones?) code? web site?
 
  Ken
 
 

--
Jacob L. Leifman email: jac...@bitwise.net
Bitwise Internet Technologies, Inc.tel: (617) 737-1837 x11
22 Drydock Avenue, Suite 310  cell: (617) 512-1024
Boston, MA 02210   fax: (617) 439-4941
--