Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Viktor Dukhovni
> On Jun 12, 2018, at 6:56 PM, Richard Levitte wrote: > > Some implementations of the iconv library take the empty string as > the locale-specific encoding, but that is in no way universal, and > isn't specified in the standard: > > http://pubs.opengroup.org/onlinepubs/009695399/functions/ico

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Richard Levitte
In message <333784c8-4870-4ddb-a892-13d552724...@dukhovni.org> on Tue, 12 Jun 2018 16:02:16 -0400, Viktor Dukhovni said: openssl-users> openssl-users> openssl-users> > On Jun 12, 2018, at 3:39 PM, Richard Levitte wrote: openssl-users> > openssl-users> >> The flags I'd like to see are: opens

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Viktor Dukhovni
> On Jun 12, 2018, at 3:39 PM, Richard Levitte wrote: > >> The flags I'd like to see are: >> >> -latin1: Passphrase is a stream of octets, each of which is a single >> unicode >> character in the range 0-255. > > I would prefer to call it -binary or something like that... it

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Richard Levitte
In message on Tue, 12 Jun 2018 11:06:40 -0400, Viktor Dukhovni said: openssl-users> openssl-users> openssl-users> > On Jun 7, 2018, at 3:40 PM, Salz, Rich wrote: openssl-users> > openssl-users> > I think you forgot that this is not what I suggested. One flag indicates it's utf-8 encoded,

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Viktor Dukhovni
> On Jun 7, 2018, at 3:40 PM, Salz, Rich wrote: > > I think you forgot that this is not what I suggested. One flag indicates > it's utf-8 encoded, don't touch it. The other flag indicates it might have > high-bit chars, don't touch it. The flags I'd like to see are: -latin1: Passphras

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Viktor Dukhovni
> On Jun 11, 2018, at 11:46 AM, Salz, Rich wrote: > > And the docs for this *new flag* explain that the behavior could change in > the future. There must be no "change in the future". Whatever flags we add now, must implement a stable interface. A flag that changes behaviour is useless. -

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Richard Levitte
In message <2418fe0a-8a61-47ad-9e60-f40bd0c79...@openssl.org> on Mon, 11 Jun 2018 19:29:09 +0200, Richard Levitte said: levitte> levitte> levitte> "Salz, Rich" skrev: (11 juni 2018 18:54:37 CEST) levitte> >>Except that, because of the way PKCS12_gen_mac() works, this isn't levitte> >

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
"Salz, Rich" skrev: (11 juni 2018 18:54:37 CEST) >>Except that, because of the way PKCS12_gen_mac() works, this isn't >true. If the input pass phrase looks like a UTF-8 encoded string > (because there are valid characters in other encodings that will like > like UTF-8 byte sequences),

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
>Except that, because of the way PKCS12_gen_mac() works, this isn't true. If the input pass phrase looks like a UTF-8 encoded string (because there are valid characters in other encodings that will like like UTF-8 byte sequences), it will be used as if -passutf8 was given inste

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message on Mon, 11 Jun 2018 16:17:33 +, Bernd Edlinger said: bernd.edlinger> So in my opinion when entering new passwords it should be restricted to bernd.edlinger> 7bit ASCII printable characters, except if advised otherwise by an bernd.edlinger> option like -pass8bit. That's Rich'

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message <5ba62036-bd2e-41b7-adf9-25c6c116e...@akamai.com> on Mon, 11 Jun 2018 16:03:48 +, "Salz, Rich" said: rsalz> >I have zero idea what the doc says, because I haven't seen the docs rsalz> yet. Did I miss the PR? rsalz> rsalz> No. It's posted here on the mailing list for d

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Bernd Edlinger
On 06/11/18 17:40, Richard Levitte wrote: > In message <8ee45344-9bfc-44f9-9db2-c384f7645...@akamai.com> on Mon, 11 Jun > 2018 15:25:23 +, "Salz, Rich" said: > > rsalz> >*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode > of > rsalz> the input pass phrase string. PKC

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
>I have zero idea what the doc says, because I haven't seen the docs yet. Did I miss the PR? No. It's posted here on the mailing list for discussion and reposted here: +=item B<-passutf8>, B<-pass8bit> + +These flags indicate the character set encoding on the password value. +By defa

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message <80d24b14-f73b-4258-b9ee-bb96f95bd...@akamai.com> on Mon, 11 Jun 2018 15:46:38 +, "Salz, Rich" said: rsalz> And the docs for this *new flag* explain that the behavior could change in the future. I have zero idea what the doc says, because I haven't seen the docs yet. Did I miss

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message <8ee45344-9bfc-44f9-9db2-c384f7645...@akamai.com> on Mon, 11 Jun 2018 15:25:23 +, "Salz, Rich" said: rsalz> >*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode of rsalz> the input pass phrase string. PKCS12_generate_mac() will then decode rsalz> rsalz> I

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
>However, what's going to happen is that PKCS12_gen_mac() will generate this for a BMPString: Which is what we do now, right? And the docs for this *new flag* explain that the behavior could change in the future. To be "pass8bit" means "pass 8bit bytes through to lower layer" But if

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message <8ee45344-9bfc-44f9-9db2-c384f7645...@akamai.com> on Mon, 11 Jun 2018 15:25:23 +, "Salz, Rich" said: rsalz> >*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode of rsalz> the input pass phrase string. PKCS12_generate_mac() will then decode rsalz> rsalz> I

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
>*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode of the input pass phrase string. PKCS12_generate_mac() will then decode I disagree. There are two reasons why users enter "illegal" passwords now, and by now requiring them to make it explicit we can (a) check only for

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message on Mon, 11 Jun 2018 15:06:01 +, "Salz, Rich" said: rsalz> > If B<-pass8bit> is given, the password is taken to be encoded in the current rsalz> > locale, but is still used directly. rsalz> > A future release might automatically convert the password to valid UTF-8 rs

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
> If B<-pass8bit> is given, the password is taken to be encoded in the current > locale, but is still used directly. > A future release might automatically convert the password to valid UTF-8 > encoding if this flag is given. I would propose that "-pass8bit" means that eac

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
And also: having *runtime* dependencies means that a pkcs12 password file generated on one system could not be used on another. I am strongly opposed to introducing that variance, especially in a "compatible" release. ___ openssl-project mailing lis

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-10 Thread Andy Polyakov
> Regarding general use of other libraries, please think carefully before > voting, 'cause this *is* tricky. If you have a look, you will see that we > *currently* depend on certain standard libraries, such as, for example, > libdl. And perhaps we should also mention the pile of libraries used w

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Kurt Roeckx
On Thu, Jun 07, 2018 at 12:07:34PM -0500, Benjamin Kaduk wrote: > On Thu, Jun 07, 2018 at 05:55:27PM +0200, Andy Polyakov wrote: > > > Regarding general use of other libraries, please think carefully before > > > voting, 'cause this *is* tricky. If you have a look, you will see that we > > > *cur

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Kurt Roeckx
On Thu, Jun 07, 2018 at 05:19:48PM +0200, Richard Levitte wrote: > Regarding general use of other libraries, please think carefully before > voting, 'cause this *is* tricky. If you have a look, you will see that we > *currently* depend on certain standard libraries, such as, for example, > libdl

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Viktor Dukhovni
> On Jun 7, 2018, at 3:59 PM, Salz, Rich wrote: > > If B<-pass8bit> is given, the password is taken to be encoded in the current > locale, but is still used directly. > A future release might automatically convert the password to valid UTF-8 > encoding if this flag is given. I would propose t

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
Here is proposed documentation for what I am suggesting. =item B<-passutf8>, B<-pass8bit> These flags indicate the character set encoding on the password value. By default, non-ASCII characters are rejected. This is new to OpenSSL 1.1.1, and is used to enforce complains with the PKCS#12 standard.

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
>So even rejecting correctly encoded utf-8? I think you forgot that this is not what I suggested. One flag indicates it's utf-8 encoded, don't touch it. The other flag indicates it might have high-bit chars, don't touch it. ___ openssl-projec

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
"Salz, Rich" skrev: (7 juni 2018 21:29:40 CEST) >>My main concern is that currently, openssl pkcs12 may create >broken pkcs12 files (because it misinterprets the pass phrase when >constructing a BMPString), and doesn't notify the user at all (doesn't >even check). > > >For those who haven'

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
>My main concern is that currently, openssl pkcs12 may create broken pkcs12 > files (because it misinterprets the pass phrase when constructing a > BMPString), and doesn't notify the user at all (doesn't even check). For those who haven't reada the PR and all its comments, I propose that

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
Viktor Dukhovni skrev: (7 juni 2018 21:16:53 CEST) >On Thu, Jun 07, 2018 at 09:01:15PM +0200, Richard Levitte wrote: > >> We don't have to answer the question "how high" now. I'm fully >> prepared to have the use of iconv limited to platforms where we know >> it's available (for example, we - o

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
> It already isn't. Depending on your regional settings, a passphrase isn't > always encoded the same way. So no, not a first. That is *different.* That is a user setting changing behavior. This would be the exact same executable, and even user settings, doing different things. _

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
"Salz, Rich" skrev: (7 juni 2018 21:09:55 CEST) >>We don't have to answer the question "how high" now. I'm fully > prepared to have the use of iconv limited to platforms where we know >it's available > >That then means that the pkcs12 program is not compatible in behavior >across plat

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Viktor Dukhovni
On Thu, Jun 07, 2018 at 09:01:15PM +0200, Richard Levitte wrote: > We don't have to answer the question "how high" now. I'm fully > prepared to have the use of iconv limited to platforms where we know > it's available (for example, we - or well, *I* - know that VMS has the > iconv API in the C RT

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
>We don't have to answer the question "how high" now. I'm fully prepared to have the use of iconv limited to platforms where we know it's available That then means that the pkcs12 program is not compatible in behavior across platforms. This would be a first, for us. __

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
In message on Thu, 7 Jun 2018 16:58:20 +0200, Andy Polyakov said: appro> One can argue that iconv was actually standardized, and in such appro> case it would be appropriate to make it conditional on appro> _POSIX_VERSION. [Though it doesn't seem to be part of pull appro> request in question. Wh

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
In message on Thu, 7 Jun 2018 11:56:00 -0400, Viktor Dukhovni said: openssl-users> openssl-users> openssl-users> > On Jun 7, 2018, at 11:19 AM, Richard Levitte wrote: openssl-users> > openssl-users> > Regarding general use of other libraries, please openssl-users> > think carefully before

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
In message on Thu, 7 Jun 2018 17:55:27 +0200, Andy Polyakov said: appro> > Regarding general use of other libraries, please think carefully before voting, 'cause this *is* tricky. If you have a look, you will see that we *currently* depend on certain standard libraries, such as, for example,

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
>Taking off on a bit of a tangent, how much justification did we go through when adding pthreads as a dependency. It's an optional, but enabled by default, dependency which is different. We had a lot of discussion within what was then openssl-team. ___

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Benjamin Kaduk
On Thu, Jun 07, 2018 at 05:55:27PM +0200, Andy Polyakov wrote: > > Regarding general use of other libraries, please think carefully before > > voting, 'cause this *is* tricky. If you have a look, you will see that we > > *currently* depend on certain standard libraries, such as, for example, > >

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Viktor Dukhovni
> On Jun 7, 2018, at 11:19 AM, Richard Levitte wrote: > > Regarding general use of other libraries, please think carefully before > voting, 'cause this *is* tricky. If you have a look, you will see that we > *currently* depend on certain standard libraries, such as, for example, > libdl. An

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Andy Polyakov
> Regarding general use of other libraries, please think carefully before > voting, 'cause this *is* tricky. If you have a look, you will see that we > *currently* depend on certain standard libraries, such as, for example, libdl. One has to recognize that each dependency has to be justified. So

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
Regarding general use of other libraries, please think carefully before voting, 'cause this *is* tricky. If you have a look, you will see that we *currently* depend on certain standard libraries, such as, for example, libdl. And perhaps we should also mention the pile of libraries used with wind

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Andy Polyakov
> This PR has been blocked, forcing a vote: > > https://github.com/openssl/openssl/pull/6392 > > Background: we have been sloppy when producing PKCS#12 files, creating > objects that aren't interoperable. This can only happen with non-UTF8 > input methods, so this PR adds a higher level of c

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
I see you already started the votes. No time for discussion? I think OpenSSL should be a "fundamental" system library. Perhaps the apps are different, but it should not require new libraries but could use them if available -- either at run-time or via config/build. I think iconv in particular