Re: Desire to contribute port for MS Windows for LibreSSL
On Wed, Oct 22, 2014 at 07:28:40PM -0500, Brent Cook wrote: On Wed, Oct 22, 2014 at 4:02 PM, Ted Unangst t...@tedunangst.com wrote: On Wed, Oct 22, 2014 at 16:07, Michael B. Trausch wrote: [snip] I think the first piece is of course the necessary build files. Access to Windows machines isn't really a problem, so sorting out the necessary ifdefs or other changes can be done on a case by case basis, but first we need the infrastructure in place to start feeding everything into the compiler. It sounds like you already have this. Makefiles? Posting those files somewhere would be a great start, and since it's entirely independent of existing code it can be integrated fairly easily even if it doesn't build without further changes. With mingw as the target, most of the code already builds with no other changes, and automake does the trick. I just built it with a mingw cross-compiler using: ./configure --host=i686-w64-mingw32 about 15 files need updates, largely in libcrypto/bio and openssl(1). This is similar to my local modifications, most of which I have yet to exhaustively test. So far I've not modified any of the automake inputs. https://gist.github.com/busterb/f36aa6de0ea47542aa41 Some parts, like b_log.c, do not seem to have much use as part of a Windows port beyond just being stubbing out. In that particular case, writing to the Windows event log, as OpenSSL does, does not have much value if you don't have an accompanying resource object built into your binary (e.g. a static library), so that is already mostly broken in OpenSSL anyway. Ah, I'd forgotten about that. Perhaps using a method of logging that delegates to the application using a callback, and on POSIX have a default handler that goes to syslog? Then, on Linux one could use a handler for the systemd journal, and on Windows the app would have to deal with it. I would think that asking the library to set the handler back to NULL would mean use platform default in this instance, but I can also see how that might not be considered a great idea. The portable libressl releases have tried to push the compatibility changes out of the main sources, confining them to isolated platform specific files, but I recognize that the Windows changes may be more extensive. I think our strategy will be to include some minor ifdefs where necessary, or to copy files in other cases. i.e., socket.c may become winsocket.c. We're trying to avoid too much interleaving within individual files. Would we possibly split files on boundaries where the common and uncommon bits are grouped more closely? I believe we did something like that with arc4random to make porting easier and reduce the number of #ifdefs to a minimum. The BoringSSL versions do have some minimal #ifdefs. My personal preference has typically been to jettison autotools (it's always a big headache even when you've been using it for 10+ years, IMHO) and use GNU make; then I simply structure things with a small receipe generator that creates rules for all platforms I support, and builds the software for all supported platforms in a single make invocation. This method wouldn't work here, of course. The important piece is this: I have 'common' source files, 'posix' source files, and 'windows' source files. I will use #ifdefs in common files only when wrapping one or two lines. I move headers to a single project-wide include so that I don't have to manage #ifdef soup in the headers for the whole project. Major point being, I'd very much like to ensure as close to full support for Windows as possible. It is my intent to be able to use LibreSSL in all of my portable projects so that I only need to worry about programming to exactly one interface. --- Mike -- Michael B. TrauschFortified Tech Systems LLC P: (404) 692-6078 or (844) GR8-TECH (Toll Free)
Desire to contribute port for MS Windows for LibreSSL
Hello,
I would like to contribute a port for the Microsoft operating systems
so that LibreSSL can support these systems without the GPL'd Cygwin
DLL being present on the system. OpenSSL works on them already, but
support for that was removed due to its apparent insecurity and
kludgyness.
I have done a very ugly port that builds and works in the few
scenarios that I've tested with, but it's not complete as some
features (mostly the ones that allow disabling at compile time) need
more work to finish porting.
The major reason that I'm writing is that I can find no really decent
example or documentation as to how to add support for the platform
to the source tree. While LibreSSL uses the GNU build system, instead
of the custom Perl-based configuration that OpenSSL uses, it seems
that the way that portability is achieved is very different from what
I am used to; at any rate, I don't really understand it. I wouldn't
dare even show the patch I've got, as I can tell you right now it
would be refused. :-)
I've been hunting around the past few days, and found not much. I
looked at the OpenSSH portable project, but the only Windows build it
seems to support is under the Cygwin runtime, which I cannot use for
various reasons.
I realize that porting to Windows is going to involve a decent amount
of work to do it in a way that will be acceptable to this project.
I'm willing to do it such that it'll make you guys happy and I'm even
willing to fork over the copyright to the contribution if so needed.
I just need a native Windows 32-bit and 64-bit version that supports
all current Microsoft-supported systems (e.g., Vista and later).
I would have zero intention of including support for Visual C++
compilers, as they really do all stink. My contribution would only
support using the {i686,x86_64}-w64-mingw32 toolchains and would
enable building of LibreSSL on POSIX systems via cross-compilation and
on Windows systems that have a native installation of the mingw64
toolchain.
Below is additional information that I've learned thus far, in case it
affects the replies in any way... if not, just ignore it.
* A native port to Windows includes using some of the functionality
in Winsock2 which isn't (presently) available in the mingw64
toolchain, such as InetPton (no, that's not mistyped, that's what
they call that function).
* Native Windows has a signal.h, but it doesn't not comply with
anything reasonable from anything I've ever seen anywhere. On
Windows, this means that signal.h must be ignored and likely the
sections which rely on handling of signals will need to have new
functions written which use the Windows API.
* Windows does not have the typical sys/socket.h, sys/ioctl.h,
netdb.h, etc. headers, instead using winsock2.h and
ws2tcpip.h. These are MOSTLY compatible with the BSD sockets
API, but there are tweaks required (inet_pton - InetPton can be
handled with a #define; Windows doesn't use SHUT_RD, SHUT_WR, or
SHUT_RDWR but rather SD_SEND, SD_RECEIVE and SD_BOTH; some APIs
have no 1:1 equivalents in the Win32 API, such as ioctl and the
like, but can be implemented with functions to enable 1:1 mapping).
* Windows doesn't have termios or anything like it.
* I haven't tested any of the sockets functionality on the build I
have, but it is very likely to not work without some extra
massaging; in Windows, the namespaces identifying file descriptors
and sockets are separate. There are methods of attempting
unification of the two, but they seem fragile.
Additionally, I would like to know if there are any preferences for
how the work is to be done and so forth. I've already reconfigured my
local emacs so that I'm able to use the style that OpenBSD already
uses in the sources, and I'll comply with existing writing style in
each file.
In essense, the current working tree I have is pretty crappy: it's
full of peppered #ifdefs and the like, just to get it to build. I
have a roadmap for completing the port, I just need to know the
OpenBSD-specific requirements for contributing the port, or be pointed
to a document which explains all of that to me.
One last item to mention: I am an American, living in the US. I would
therefore be able to provide diffs only, in order to avoid our export
controls on cryptographic software. I would not be making any
alterations to cryptographic code, and only exporting unified diffs
which enable the existing system to build on Windows. If this is a
blocker to being able to contribute to the project, it'd be good to
know that now, and I'll just maintain a local fork or something. I am
not a lawyer, but after reading through what the controls are, I'm
relatively confident that diffs which do not themselves contain
cryptographic functionality are exportable without issue.
Thanks!
Michael B. Trausch
--
Michael B. TrauschFortified Tech Systems LLC
P:
Re: Desire to contribute port for MS Windows for LibreSSL
On Wed, Oct 22, 2014 at 16:07, Michael B. Trausch wrote: I have done a very ugly port that builds and works in the few scenarios that I've tested with, but it's not complete as some features (mostly the ones that allow disabling at compile time) need more work to finish porting. I think the first piece is of course the necessary build files. Access to Windows machines isn't really a problem, so sorting out the necessary ifdefs or other changes can be done on a case by case basis, but first we need the infrastructure in place to start feeding everything into the compiler. It sounds like you already have this. Makefiles? Posting those files somewhere would be a great start, and since it's entirely independent of existing code it can be integrated fairly easily even if it doesn't build without further changes. The portable libressl releases have tried to push the compatibility changes out of the main sources, confining them to isolated platform specific files, but I recognize that the Windows changes may be more extensive. I think our strategy will be to include some minor ifdefs where necessary, or to copy files in other cases. i.e., socket.c may become winsocket.c. We're trying to avoid too much interleaving within individual files.
Re: Desire to contribute port for MS Windows for LibreSSL
On Wed, Oct 22, 2014 at 4:02 PM, Ted Unangst t...@tedunangst.com wrote: On Wed, Oct 22, 2014 at 16:07, Michael B. Trausch wrote: I have done a very ugly port that builds and works in the few scenarios that I've tested with, but it's not complete as some features (mostly the ones that allow disabling at compile time) need more work to finish porting. I think the first piece is of course the necessary build files. Access to Windows machines isn't really a problem, so sorting out the necessary ifdefs or other changes can be done on a case by case basis, but first we need the infrastructure in place to start feeding everything into the compiler. It sounds like you already have this. Makefiles? Posting those files somewhere would be a great start, and since it's entirely independent of existing code it can be integrated fairly easily even if it doesn't build without further changes. With mingw as the target, most of the code already builds with no other changes, and automake does the trick. I just built it with a mingw cross-compiler using: ./configure --host=i686-w64-mingw32 about 15 files need updates, largely in libcrypto/bio and openssl(1). https://gist.github.com/busterb/f36aa6de0ea47542aa41 Some parts, like b_log.c, do not seem to have much use as part of a Windows port beyond just being stubbing out. In that particular case, writing to the Windows event log, as OpenSSL does, does not have much value if you don't have an accompanying resource object built into your binary (e.g. a static library), so that is already mostly broken in OpenSSL anyway. If we wanted to be slightly more cavalier and adopt something close to BoringSSL's bio subsystem (adding back the parts it jettisoned for compatibility), there is already reasonable Windows support there, IMO. The portable libressl releases have tried to push the compatibility changes out of the main sources, confining them to isolated platform specific files, but I recognize that the Windows changes may be more extensive. I think our strategy will be to include some minor ifdefs where necessary, or to copy files in other cases. i.e., socket.c may become winsocket.c. We're trying to avoid too much interleaving within individual files. Would we possibly split files on boundaries where the common and uncommon bits are grouped more closely? I believe we did something like that with arc4random to make porting easier and reduce the number of #ifdefs to a minimum. The BoringSSL versions do have some minimal #ifdefs.
Re: Desire to contribute port for MS Windows for LibreSSL
On Wed, Oct 22, 2014 at 3:07 PM, Michael B. Trausch
m...@fortifiedtechsystems.com wrote:
Hello,
I would like to contribute a port for the Microsoft operating systems
so that LibreSSL can support these systems without the GPL'd Cygwin
DLL being present on the system. OpenSSL works on them already, but
support for that was removed due to its apparent insecurity and
kludgyness.
I have done a very ugly port that builds and works in the few
scenarios that I've tested with, but it's not complete as some
features (mostly the ones that allow disabling at compile time) need
more work to finish porting.
The major reason that I'm writing is that I can find no really decent
example or documentation as to how to add support for the platform
to the source tree. While LibreSSL uses the GNU build system, instead
of the custom Perl-based configuration that OpenSSL uses, it seems
that the way that portability is achieved is very different from what
I am used to; at any rate, I don't really understand it. I wouldn't
dare even show the patch I've got, as I can tell you right now it
would be refused. :-)
Well, if the platform is mostly POSIX, it is relatively easier. For
reference, here is a work-in-progress HP-UX port:
https://github.com/kinichiro/portable/commit/ee8dadee8f52e56eb528d3fc372cea7024ae96f6
I've been hunting around the past few days, and found not much. I
looked at the OpenSSH portable project, but the only Windows build it
seems to support is under the Cygwin runtime, which I cannot use for
various reasons.
I certainly could share how to build it under mingw, though take count that
as an endorsement :)
I realize that porting to Windows is going to involve a decent amount
of work to do it in a way that will be acceptable to this project.
I'm willing to do it such that it'll make you guys happy and I'm even
willing to fork over the copyright to the contribution if so needed.
I just need a native Windows 32-bit and 64-bit version that supports
all current Microsoft-supported systems (e.g., Vista and later).
I would have zero intention of including support for Visual C++
compilers, as they really do all stink. My contribution would only
support using the {i686,x86_64}-w64-mingw32 toolchains and would
enable building of LibreSSL on POSIX systems via cross-compilation and
on Windows systems that have a native installation of the mingw64
toolchain.
Below is additional information that I've learned thus far, in case it
affects the replies in any way... if not, just ignore it.
* A native port to Windows includes using some of the functionality
in Winsock2 which isn't (presently) available in the mingw64
toolchain, such as InetPton (no, that's not mistyped, that's what
they call that function).
* Native Windows has a signal.h, but it doesn't not comply with
anything reasonable from anything I've ever seen anywhere. On
Windows, this means that signal.h must be ignored and likely the
sections which rely on handling of signals will need to have new
functions written which use the Windows API.
These are only in the openssl(1) application, right? I wonder if we just
started by focusing on the libraries if any Windows user would really care
about openssl(1) initially. It certainly doesn't scratch any itch I have
either.
* Windows does not have the typical sys/socket.h, sys/ioctl.h,
netdb.h, etc. headers, instead using winsock2.h and
ws2tcpip.h. These are MOSTLY compatible with the BSD sockets
API, but there are tweaks required (inet_pton - InetPton can be
handled with a #define; Windows doesn't use SHUT_RD, SHUT_WR, or
SHUT_RDWR but rather SD_SEND, SD_RECEIVE and SD_BOTH; some APIs
have no 1:1 equivalents in the Win32 API, such as ioctl and the
like, but can be implemented with functions to enable 1:1 mapping).
* Windows doesn't have termios or anything like it.
* I haven't tested any of the sockets functionality on the build I
have, but it is very likely to not work without some extra
massaging; in Windows, the namespaces identifying file descriptors
and sockets are separate. There are methods of attempting
unification of the two, but they seem fragile.
There's also the the yucky subject of sockets in 64-bit Windows not being
an 'int', but a pointer instead:
typedef UINT_PTRSOCKET;
and a pox on whoever invented 'closesocket()'
Additionally, I would like to know if there are any preferences for
how the work is to be done and so forth. I've already reconfigured my
local emacs so that I'm able to use the style that OpenBSD already
uses in the sources, and I'll comply with existing writing style in
each file.
In essense, the current working tree I have is pretty crappy: it's
full of peppered #ifdefs and the like, just to get it to build. I
have a roadmap for completing the port, I just need to know the
OpenBSD-specific requirements for contributing the port,
Re: Desire to contribute port for MS Windows for LibreSSL
On Wed, Oct 22, 2014 at 7:51 PM, Brent Cook bust...@gmail.com wrote: I've been hunting around the past few days, and found not much. I looked at the OpenSSH portable project, but the only Windows build it seems to support is under the Cygwin runtime, which I cannot use for various reasons. I certainly could share how to build it under mingw, though take count that as an endorsement :) Ahem: 'don't take this as an endorsement', but if it useful for comparison purposes, variants of: ./Configure [your favorite no-* flags] mingw / mingw64 works for me for building static libraries.
