Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Viktor Dukhovni



> On Jun 12, 2018, at 6:56 PM, Richard Levitte  wrote:
> 
> Some implementations of the iconv library take the empty string as
> the locale-specific encoding, but that is in no way universal, and
> isn't specified in the standard:
> 
> http://pubs.opengroup.org/onlinepubs/009695399/functions/iconv_open.html
> 
> Using nl_langinfo() to get the locale-specific encoding will, as far
> as I know, always get you what you expect.

On FreeBSD, after (required) calling:

setlocale(LC_CTYPE, "");

The nl_langinfo(CODESET) returns the correct charset for by
UTF-8 terminal emulator for which my environment has:

LC_CTYPE=en_US.UTF-8

With that, iconv_open() and iconv() behave correctly converting
to from ISO-8859-1 and UTF-8 (minimal tests).  Without the
setlocale() call, my encoding is always US-ASCII, and iconv
is naturally crippled.

-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Richard Levitte
In message <333784c8-4870-4ddb-a892-13d552724...@dukhovni.org> on Tue, 12 Jun 
2018 16:02:16 -0400, Viktor Dukhovni  said:

openssl-users> 
openssl-users> 
openssl-users> > On Jun 12, 2018, at 3:39 PM, Richard Levitte 
 wrote:
openssl-users> > 
openssl-users> >> The flags I'd like to see are:
openssl-users> >> 
openssl-users> >>   -latin1:  Passphrase is a stream of octets, each of which 
is a single unicode
openssl-users> >> character in the range 0-255.
openssl-users> > 
openssl-users> > I would prefer to call it -binary or something like that...  it
openssl-users> > certainly comes down to the same thing in practice, and should
openssl-users> > translate exactly to the pre-1.1.0 behaviour.
openssl-users> 
openssl-users> I won't quibble over the name.
openssl-users> 
openssl-users> > 
openssl-users> >>   -utf8:Passphrase is already utf-8 encoded
openssl-users> >> 
openssl-users> >>   -ascii:   Passphrase must be ASCII, reject inadvertent 
8-bit input.
openssl-users> > 
openssl-users> > ... and if none of these are given?
openssl-users> 
openssl-users> Not sure.  We could opt for "-binary" by default, which is 
backwards
openssl-users> compatible, but it produces non-standard outputs, which is a 
disfavour
openssl-users> to new users.  We could go with "-ascii" as a default, forcing 
failure
openssl-users> for non-ascii passwords without an explicit indication of 
encoding.
openssl-users> The second seems more appealing to me.

Same here.

openssl-users> >> And as available:
openssl-users> >> 
openssl-users> >>   -toutf8:   Convert passphrase from the input encoding to 
UTF-8.
openssl-users> >>Either using the locale-specific encoding, or yet
openssl-users> >>  another flag:
openssl-users> >> 
openssl-users> >>   -encoding: A platform-specific name for the input encoding 
understood
openssl-users> >>  by the system's encoding conversion library 
(iconv on Unix).
openssl-users> > 
openssl-users> > If the availability of -toutf8 depends on the presumed 
presence of
openssl-users> > iconv(), then we can assume that nl_langinfo() is present as 
well.
openssl-users> > That renders -encoding unnecessary, unless you want to use it 
to
openssl-users> > override the locale-specific encoding.
openssl-users> 
openssl-users> The purpose is specifically to override the encoding when it is 
wrong
openssl-users> for some reason.  The iconv library takes the empty string as the
openssl-users> locale-specific encoding, so we should not need nl_langinfo(), 
unless
openssl-users> that's known to produce better results.

Some implementations of the iconv library take the empty string as
the locale-specific encoding, but that is in no way universal, and
isn't specified in the standard:

http://pubs.opengroup.org/onlinepubs/009695399/functions/iconv_open.html

Using nl_langinfo() to get the locale-specific encoding will, as far
as I know, always get you what you expect.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Viktor Dukhovni



> On Jun 12, 2018, at 3:39 PM, Richard Levitte  wrote:
> 
>> The flags I'd like to see are:
>> 
>>   -latin1:  Passphrase is a stream of octets, each of which is a single 
>> unicode
>> character in the range 0-255.
> 
> I would prefer to call it -binary or something like that...  it
> certainly comes down to the same thing in practice, and should
> translate exactly to the pre-1.1.0 behaviour.

I won't quibble over the name.

> 
>>   -utf8:Passphrase is already utf-8 encoded
>> 
>>   -ascii:   Passphrase must be ASCII, reject inadvertent 8-bit input.
> 
> ... and if none of these are given?

Not sure.  We could opt for "-binary" by default, which is backwards
compatible, but it produces non-standard outputs, which is a disfavour
to new users.  We could go with "-ascii" as a default, forcing failure
for non-ascii passwords without an explicit indication of encoding.
The second seems more appealing to me.

>> And as available:
>> 
>>   -toutf8:   Convert passphrase from the input encoding to UTF-8.
>>   Either using the locale-specific encoding, or yet
>>  another flag:
>> 
>>   -encoding: A platform-specific name for the input encoding understood
>>  by the system's encoding conversion library (iconv on Unix).
> 
> If the availability of -toutf8 depends on the presumed presence of
> iconv(), then we can assume that nl_langinfo() is present as well.
> That renders -encoding unnecessary, unless you want to use it to
> override the locale-specific encoding.

The purpose is specifically to override the encoding when it is wrong
for some reason.  The iconv library takes the empty string as the
locale-specific encoding, so we should not need nl_langinfo(), unless
that's known to produce better results.

-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Richard Levitte
In message  on Tue, 12 Jun 
2018 11:06:40 -0400, Viktor Dukhovni  said:

openssl-users> 
openssl-users> 
openssl-users> > On Jun 7, 2018, at 3:40 PM, Salz, Rich  
wrote:
openssl-users> > 
openssl-users> > I think you forgot that this is not what I suggested.  One 
flag indicates it's utf-8 encoded, don't touch it.  The other flag indicates it 
might have high-bit chars, don't touch it.
openssl-users> 
openssl-users> The flags I'd like to see are:
openssl-users> 
openssl-users>   -latin1:  Passphrase is a stream of octets, each of which is a 
single unicode
openssl-users> character in the range 0-255.

I would prefer to call it -binary or something like that...  it
certainly comes down to the same thing in practice, and should
translate exactly to the pre-1.1.0 behaviour.

openssl-users>   -utf8:Passphrase is already utf-8 encoded
openssl-users> 
openssl-users>   -ascii:   Passphrase must be ASCII, reject inadvertent 8-bit 
input.

... and if none of these are given?

openssl-users> And as available:
openssl-users> 
openssl-users>   -toutf8:   Convert passphrase from the input encoding to UTF-8.
openssl-users>   Either using the locale-specific encoding, or yet
openssl-users>  another flag:
openssl-users> 
openssl-users>   -encoding: A platform-specific name for the input encoding 
understood
openssl-users>  by the system's encoding conversion library (iconv 
on Unix).

If the availability of -toutf8 depends on the presumed presence of
iconv(), then we can assume that nl_langinfo() is present as well.
That renders -encoding unnecessary, unless you want to use it to
override the locale-specific encoding.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Viktor Dukhovni



> On Jun 7, 2018, at 3:40 PM, Salz, Rich  wrote:
> 
> I think you forgot that this is not what I suggested.  One flag indicates 
> it's utf-8 encoded, don't touch it.  The other flag indicates it might have 
> high-bit chars, don't touch it.

The flags I'd like to see are:

  -latin1:  Passphrase is a stream of octets, each of which is a single unicode
character in the range 0-255.

  -utf8:Passphrase is already utf-8 encoded

  -ascii:   Passphrase must be ASCII, reject inadvertent 8-bit input.

And as available:

  -toutf8:   Convert passphrase from the input encoding to UTF-8.
 Either using the locale-specific encoding, or yet
 another flag:

  -encoding: A platform-specific name for the input encoding understood
 by the system's encoding conversion library (iconv on Unix).

None of these flags change semantics after introduction.

-- 
-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Viktor Dukhovni



> On Jun 11, 2018, at 11:46 AM, Salz, Rich  wrote:
> 
> And the docs for this *new flag* explain that the behavior could change in 
> the future.

There must be no "change in the future".  Whatever flags
we add now, must implement a stable interface.  A flag
that changes behaviour is useless.

-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-12 Thread Richard Levitte
In message <2418fe0a-8a61-47ad-9e60-f40bd0c79...@openssl.org> on Mon, 11 Jun 
2018 19:29:09 +0200, Richard Levitte  said:

levitte> 
levitte> 
levitte> "Salz, Rich"  skrev: (11 juni 2018 18:54:37 CEST)
levitte> >>Except that, because of the way PKCS12_gen_mac() works, this 
isn't
levitte> >true.  If the input pass phrase looks like a UTF-8 encoded string
levitte> >  (because there are valid characters in other encodings that will 
like
levitte> >  like UTF-8 byte sequences), it will be used as if -passutf8 was 
given
levitte> >instead.
levitte> >  
levitte> >Well, at least I started down the path.  I wonder if one of those 
flags
levitte> >should set the keygen to asc?
levitte> 
levitte> That could be an idea, except that's not an easy change. Effectively, 
that's exactly equivalent to doing a naïve utf-8 encode in the application. 

BTW, this subthread is a development discussion...  shouldn't that be
happening on github, along with a WIP PR?

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte


"Salz, Rich"  skrev: (11 juni 2018 18:54:37 CEST)
>>Except that, because of the way PKCS12_gen_mac() works, this isn't
>true.  If the input pass phrase looks like a UTF-8 encoded string
>  (because there are valid characters in other encodings that will like
>  like UTF-8 byte sequences), it will be used as if -passutf8 was given
>instead.
>  
>Well, at least I started down the path.  I wonder if one of those flags
>should set the keygen to asc?

That could be an idea, except that's not an easy change. Effectively, that's 
exactly equivalent to doing a naïve utf-8 encode in the application. 

Cheers 
Richard 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
>Except that, because of the way PKCS12_gen_mac() works, this isn't
true.  If the input pass phrase looks like a UTF-8 encoded string
(because there are valid characters in other encodings that will like
like UTF-8 byte sequences), it will be used as if -passutf8 was given
instead.
  
Well, at least I started down the path.  I wonder if one of those flags should 
set the keygen to asc?

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message 

 on Mon, 11 Jun 2018 16:17:33 +, Bernd Edlinger  
said:

bernd.edlinger> So in my opinion when entering new passwords it should be 
restricted to
bernd.edlinger> 7bit ASCII printable characters, except if advised otherwise by 
an
bernd.edlinger> option like -pass8bit.

That's Rich's intent, and I'm fine with that.  It's the fine print of
what message we tell with -pass8bit that's being disputed.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message <5ba62036-bd2e-41b7-adf9-25c6c116e...@akamai.com> on Mon, 11 Jun 
2018 16:03:48 +, "Salz, Rich"  said:

rsalz> >I have zero idea what the doc says, because I haven't seen the docs
rsalz> yet.  Did I miss the PR?
rsalz>   
rsalz> No.  It's posted here on the mailing list for discussion and reposted 
here:

Ah, found it in the archive...  I somehow glossed over it before,
sorry.

rsalz> +If B<-pass8bit> is given, the password is taken to be encoded in the 
current
rsalz> +locale, but is still used directly; note that
rsalz> +a future release might automatically convert the password to valid UTF-8
rsalz> +encoding if this flag is given.

Except that, because of the way PKCS12_gen_mac() works, this isn't
true.  If the input pass phrase looks like a UTF-8 encoded string
(because there are valid characters in other encodings that will like
like UTF-8 byte sequences), it will be used as if -passutf8 was given
instead.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Bernd Edlinger
On 06/11/18 17:40, Richard Levitte wrote:
> In message <8ee45344-9bfc-44f9-9db2-c384f7645...@akamai.com> on Mon, 11 Jun 
> 2018 15:25:23 +, "Salz, Rich"  said:
> 
> rsalz> >*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode 
> of
> rsalz> the input pass phrase string.  PKCS12_generate_mac() will then 
> decode
> rsalz>
> rsalz> I disagree.
> rsalz>
> rsalz> There are two reasons why users enter "illegal" passwords now,
> rsalz> and by now requiring them to make it explicit we can (a) check
> rsalz> only for ASCII on current inputs; (b) make them thing about
> rsalz> what they're doing and require them to specify; (c) set the
> rsalz> expectation that something will change in the future.
> 
> [btw, PKCS12_gen_mac(), not PKCS12_generate_mac()]
> 
> So wait, if the user enters this:
> 
>  openssl pkcs12 -export -in foo.pem -out foo.p12 \
>  -pass8bit -password pass:`echo 72c3a46b61 | xxd -r -p`
> 
> ...  then it seems "natural" that the user would expect the resulting
> BMPString to become this set of bytes, right?
> 
>  0x00, 0x72, 0x00, 0xc3, 0x00, 0xa4, 0x00, 0x6b, 0x00, 0x61, 0x00, 0x00
> 
> However, what's going to happen is that PKCS12_gen_mac() will generate
> this for a BMPString:
> 
>  0x00, 0x72, 0x00, 0xe4, 0x00, 0x6b, 0x00, 0x61, 0x00, 0x00
> 
> Why?  Because the input pass phrase can be interpreted as a UTF-8
> encoded string, and PKCS12_gen_mac() will decode it thusly.
> 
>  From a user interface point of view, I would fine such behavior very
> surprising, and not at all what I'd expect for a flag named '-pass8bit'
> 

I think there are many ways for the user to shoot into his own knee with
entering unicode glyphs accidentally, with are even invisible when
printed to the console, just think of the EM DASH U+2014: "—"
Or unicode non break space U+00A0 which looks like an ordinary space but
is something different

As a user, I would not be happy if one of these slipped into a password,
that's certainly sure.

So in my opinion when entering new passwords it should be restricted to
7bit ASCII printable characters, except if advised otherwise by an
option like -pass8bit.


Bernd.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
>I have zero idea what the doc says, because I haven't seen the docs
yet.  Did I miss the PR?
  
No.  It's posted here on the mailing list for discussion and reposted here:


+=item B<-passutf8>, B<-pass8bit>
+
+These flags indicate the character set encoding on the password value.
+By default, non-ASCII characters are rejected. This is new behavior
+with OpenSSL 1.1.1,
+and is used to enforce compliance with the PKCS#12 standard.
+If B<-passutf8> is given, then the password is taken to be valid UTF-8 
encoding,
+and will be used directly.
+If B<-pass8bit> is given, the password is taken to be encoded in the current
+locale, but is still used directly; note that
+a future release might automatically convert the password to valid UTF-8
+encoding if this flag is given.
+


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message <80d24b14-f73b-4258-b9ee-bb96f95bd...@akamai.com> on Mon, 11 Jun 
2018 15:46:38 +, "Salz, Rich"  said:

rsalz> And the docs for this *new flag* explain that the behavior could change 
in the future.

I have zero idea what the doc says, because I haven't seen the docs
yet.  Did I miss the PR?

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message <8ee45344-9bfc-44f9-9db2-c384f7645...@akamai.com> on Mon, 11 Jun 
2018 15:25:23 +, "Salz, Rich"  said:

rsalz> >*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode of
rsalz> the input pass phrase string.  PKCS12_generate_mac() will then decode
rsalz>   
rsalz> I disagree.
rsalz> 
rsalz> There are two reasons why users enter "illegal" passwords now, and by 
now requiring them to make it explicit we can (a) check only for ASCII on 
current inputs; (b) make them thing about what they're doing and require them 
to specify; (c) set the expectation that something will change in the future.

A variant is to check if the 8bit string can be decoded as a UTF-8
string and warn the user that such string is going to get screwed.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
>However, what's going to happen is that PKCS12_gen_mac() will generate
this for a BMPString:
  
Which is what we do now, right?

And the docs for this *new flag* explain that the behavior could change in the 
future.

To be "pass8bit" means "pass 8bit bytes through to lower layer"  But if we want 
to bikeshed about the name of the flag, fine.


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message <8ee45344-9bfc-44f9-9db2-c384f7645...@akamai.com> on Mon, 11 Jun 
2018 15:25:23 +, "Salz, Rich"  said:

rsalz> >*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode of
rsalz> the input pass phrase string.  PKCS12_generate_mac() will then decode
rsalz>   
rsalz> I disagree.
rsalz> 
rsalz> There are two reasons why users enter "illegal" passwords now,
rsalz> and by now requiring them to make it explicit we can (a) check
rsalz> only for ASCII on current inputs; (b) make them thing about
rsalz> what they're doing and require them to specify; (c) set the
rsalz> expectation that something will change in the future.

[btw, PKCS12_gen_mac(), not PKCS12_generate_mac()]

So wait, if the user enters this:

openssl pkcs12 -export -in foo.pem -out foo.p12 \
-pass8bit -password pass:`echo 72c3a46b61 | xxd -r -p`

...  then it seems "natural" that the user would expect the resulting
BMPString to become this set of bytes, right?

0x00, 0x72, 0x00, 0xc3, 0x00, 0xa4, 0x00, 0x6b, 0x00, 0x61, 0x00, 0x00

However, what's going to happen is that PKCS12_gen_mac() will generate
this for a BMPString:

0x00, 0x72, 0x00, 0xe4, 0x00, 0x6b, 0x00, 0x61, 0x00, 0x00

Why?  Because the input pass phrase can be interpreted as a UTF-8
encoded string, and PKCS12_gen_mac() will decode it thusly.

>From a user interface point of view, I would fine such behavior very
surprising, and not at all what I'd expect for a flag named '-pass8bit'

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
>*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode of
the input pass phrase string.  PKCS12_generate_mac() will then decode
  
I disagree.

There are two reasons why users enter "illegal" passwords now, and by now 
requiring them to make it explicit we can (a) check only for ASCII on current 
inputs; (b) make them thing about what they're doing and require them to 
specify; (c) set the expectation that something will change in the future.


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Richard Levitte
In message  on Mon, 11 Jun 
2018 15:06:01 +, "Salz, Rich"  said:

rsalz> > If B<-pass8bit> is given, the password is taken to be encoded in 
the current
rsalz> > locale, but is still used directly.
rsalz> > A future release might automatically convert the password to valid 
UTF-8
rsalz> > encoding if this flag is given.
rsalz> 
rsalz> I would propose that "-pass8bit" means that each byte of the input is
rsalz> a unicode code point value (i.e. ASCII or LATIN1 supplement) and 
we'll
rsalz> convert to UCS-2 by prepending 0x00 to each one.  If so, I would 
expect
rsalz> this flag to NOT ever change its meaning.
rsalz> 
rsalz> I don't see the point of this.
rsalz> 
rsalz> My goal, with the two flags, was to allow users to make explicit what 
they want, and to warn them that *one* of the cases might/will change in the 
future.

Well, that is what's done in PKCS12_generate_mac(), so this isn't
something that should be done by the application.  What the appication
*must* do when getting '-pass8bit' is to do a naïve UTF-8 encode of
the input pass phrase string.  PKCS12_generate_mac() will then decode
it and zero extend every resulting byte to 16 bits.  If you *don't* do
this, you risk having any byte sequence that looks like UTF-8 in the
original input to be decoded and made into something other than what
the user intended.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
> If B<-pass8bit> is given, the password is taken to be encoded in the 
current
> locale, but is still used directly.
> A future release might automatically convert the password to valid UTF-8
> encoding if this flag is given.

I would propose that "-pass8bit" means that each byte of the input is
a unicode code point value (i.e. ASCII or LATIN1 supplement) and we'll
convert to UCS-2 by prepending 0x00 to each one.  If so, I would expect
this flag to NOT ever change its meaning.

I don't see the point of this.

My goal, with the two flags, was to allow users to make explicit what they 
want, and to warn them that *one* of the cases might/will change in the future.


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-11 Thread Salz, Rich
And also: having *runtime* dependencies means that a pkcs12 password file 
generated on one system could not be used on another.  I am strongly opposed to 
introducing that variance, especially in a "compatible" release.
 

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-10 Thread Andy Polyakov
> Regarding general use of other libraries, please think carefully before 
> voting, 'cause this *is* tricky. If you have a look, you will see that we 
> *currently* depend on certain standard libraries, such as, for example, 
> libdl. And perhaps we should also mention the pile of libraries used with 
> windows.
> 
> In my mind, this makes that more general vote ridiculous,

It certainly is, the vote should not be about such general principle.
It's way too general and most importantly *too natural* to vote about.
In sense that there are things that can't be voting matter. For example
"humans need air to breath" is not, and so is assertion that OpenSSL
needs libraries. And it's all but utterly natural to *minimize*
dependencies and make *minimum* assumptions. This means that one can
only vote on specifics, i.e. do we want specific dependency (at specific
level) or make specific assumption (at specific level). That's why
ballot should not be formulated "such as iconv", but *be* about iconv.

As for iconv per se. One has to recognize that it was standardized by
POSIX and one can expect it to be available on compliant system. Trick
is to reliably identify the said systems, and possibly by version, not
just by being X. So it's as important to have a fall-back to handle the
exclusions. As there will be exclusions. I can even name one already,
Android. And once again, I argue that there is even timing issue. It's
not right moment to make too broad assumptions about iconv availability
arguing that one is prepared to deal with issues. One can only argue
that it might be appropriate to enable it in cases one can actually
verify and confirm.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Kurt Roeckx
On Thu, Jun 07, 2018 at 12:07:34PM -0500, Benjamin Kaduk wrote:
> On Thu, Jun 07, 2018 at 05:55:27PM +0200, Andy Polyakov wrote:
> > > Regarding general use of other libraries, please think carefully before 
> > > voting, 'cause this *is* tricky. If you have a look, you will see that we 
> > > *currently* depend on certain standard libraries, such as, for example, 
> > > libdl.
> > 
> > One has to recognize that each dependency has to be justified. Sometimes
> > you don't have a choice. For example if you want to talk network on
> > Solaris, you have to link with -lsocket -lnsl. It's just part of the
> > game. But in cases one has a choice, well, one has a choice to *make*.
> > And key question is how do you anchor it. It's only natural to have as
> > little dependencies as possible, so question is what would justify extra
> > dependency.
> 
> Taking off on a bit of a tangent, how much justification did we go
> through when adding pthreads as a dependency.  I have not been
> following very much (Kurt would know more), but apparently in Debian
> there are some issues regarding (statically linked?) applications
> and libraries that use libcrypto but do not explicitly link with
> -pthread.  "Issues" here being, IIRC, crashes at runtime.

I haven't really followed it, I just saw some mentionng of it on
IRC. At least static linkig glibc itself is complicated. I guess
it's also complicated because libc itself contains stubs for the
pthread functions, so at least the order of the libraries will be
important when linking staticly. But I didn't read anything about
crashes, I might have missed things.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Kurt Roeckx
On Thu, Jun 07, 2018 at 05:19:48PM +0200, Richard Levitte wrote:
> Regarding general use of other libraries, please think carefully before 
> voting, 'cause this *is* tricky. If you have a look, you will see that we 
> *currently* depend on certain standard libraries, such as, for example, 
> libdl. And perhaps we should also mention the pile of libraries used with 
> windows.

We also started depending on pthread in 1.1.0 ...


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Viktor Dukhovni



> On Jun 7, 2018, at 3:59 PM, Salz, Rich  wrote:
> 
> If B<-pass8bit> is given, the password is taken to be encoded in the current
> locale, but is still used directly.
> A future release might automatically convert the password to valid UTF-8
> encoding if this flag is given.

I would propose that "-pass8bit" means that each byte of the input is
a unicode code point value (i.e. ASCII or LATIN1 supplement) and we'll
convert to UCS-2 by prepending 0x00 to each one.  If so, I would expect
this flag to NOT ever change its meaning.

We may internally convert to UTF-8 and then to UTF-16 largely undoing
the first conversion, but that just internal API gymnastics, not user-
observable.

-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
Here is proposed documentation for what I am suggesting.

=item B<-passutf8>, B<-pass8bit>

These flags indicate the character set encoding on the password value.
By default, non-ASCII characters are rejected. This is new to OpenSSL 1.1.1,
and is used to enforce complains with the PKCS#12 standard.
If B<-passutf8> is given, then the password is taken to be valid UTF-8 encoding,
and will be used directly.
If B<-pass8bit> is given, the password is taken to be encoded in the current
locale, but is still used directly.
A future release might automatically convert the password to valid UTF-8
encoding if this flag is given.


On 6/7/18, 3:42 PM, "Salz, Rich"  wrote:

>So even rejecting correctly encoded utf-8?
  
I think you forgot that this is not what I suggested.  One flag indicates 
it's utf-8 encoded, don't touch it.  The other flag indicates it might have 
high-bit chars, don't touch it.


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
>So even rejecting correctly encoded utf-8?
  
I think you forgot that this is not what I suggested.  One flag indicates it's 
utf-8 encoded, don't touch it.  The other flag indicates it might have high-bit 
chars, don't touch it.


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte


"Salz, Rich"  skrev: (7 juni 2018 21:29:40 CEST)
>>My main concern is that currently, openssl pkcs12 may create
>broken pkcs12 files (because it misinterprets the pass phrase when
>constructing a BMPString), and doesn't notify the user at all (doesn't
>even check). 
> 
>
>For those who haven't reada the PR and all its comments, I propose that
>we reject non-ASCII input unless one of two flags is set.  This
>prevents us from unknowingly making the situation worse, and does not
>break interop with our installed base.

So even rejecting correctly encoded utf-8?

-- 
Skickat från min Android-enhet med K-9 Mail. Ursäkta min fåordighet.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
>My main concern is that currently, openssl pkcs12 may create broken pkcs12 
> files (because it misinterprets the pass phrase when constructing a 
> BMPString), and doesn't notify the user at all (doesn't even check). 
 

For those who haven't reada the PR and all its comments, I propose that we 
reject non-ASCII input unless one of two flags is set.  This prevents us from 
unknowingly making the situation worse, and does not break interop with our 
installed base.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte


Viktor Dukhovni  skrev: (7 juni 2018 21:16:53 CEST)
>On Thu, Jun 07, 2018 at 09:01:15PM +0200, Richard Levitte wrote:
>
>> We don't have to answer the question "how high" now.  I'm fully
>> prepared to have the use of iconv limited to platforms where we know
>> it's available (for example, we - or well, *I* - know that VMS has
>the
>> iconv API in the C RTL, not even any need to link with any extra
>> library...  and we *know* it's available in glibc since version
>> 2.something).  I'm fully prepared to have to deal with people saying
>> "hey, we have that too!" and having to edit config targets as we go.
>> I do not expect any support of iconv to cover more than what we can
>> test or get patches for, as with anything else.
>
>There's also apparently some variation in the iconv API function
>prototypes (possibly "const char **" vs. "char **").  So some
>platform-dependent casting may be required...
>
>My peers suggest that this late in the release cycle, we leave the
>responsibility of ensuring UTF-8 input to the user.  We could
>describe work-arounds in documentation.  Personally, if this is
>off by default, and requires a new option to enable, and is just
>in openssl(1) and not libcrypto, I'm disinclined to say "no" until
>I see a PR (with documentation) that we can decide to leave for
>post 1.1.1 or adopt.

Please review doc/man7/passphrase-encoding.pod

My main concern is that currently, openssl pkcs12 may create broken pkcs12 
files (because it misinterprets the pass phrase when constructing a BMPString), 
and doesn't notify the user at all (doesn't even check). 

Cheers 
Richard 

-- 
Skickat från min Android-enhet med K-9 Mail. Ursäkta min fåordighet.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
> It already isn't. Depending on your regional settings, a passphrase isn't 
> always encoded the same way. 
So no, not a first. 
   
That is *different.*  That is a user setting changing behavior.  This would be 
the exact same executable, and even user settings, doing different things.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte


"Salz, Rich"  skrev: (7 juni 2018 21:09:55 CEST)
>>We don't have to answer the question "how high" now.  I'm fully
>   prepared to have the use of iconv limited to platforms where we know
>it's available
>
>That then means that the pkcs12 program is not compatible in behavior
>across platforms.  This would be a first, for us.

It already isn't. Depending on your regional settings, a passphrase isn't 
always encoded the same way. 
So no, not a first. 

-- 
Skickat från min Android-enhet med K-9 Mail. Ursäkta min fåordighet.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Viktor Dukhovni
On Thu, Jun 07, 2018 at 09:01:15PM +0200, Richard Levitte wrote:

> We don't have to answer the question "how high" now.  I'm fully
> prepared to have the use of iconv limited to platforms where we know
> it's available (for example, we - or well, *I* - know that VMS has the
> iconv API in the C RTL, not even any need to link with any extra
> library...  and we *know* it's available in glibc since version
> 2.something).  I'm fully prepared to have to deal with people saying
> "hey, we have that too!" and having to edit config targets as we go.
> I do not expect any support of iconv to cover more than what we can
> test or get patches for, as with anything else.

There's also apparently some variation in the iconv API function
prototypes (possibly "const char **" vs. "char **").  So some
platform-dependent casting may be required...

My peers suggest that this late in the release cycle, we leave the
responsibility of ensuring UTF-8 input to the user.  We could
describe work-arounds in documentation.  Personally, if this is
off by default, and requires a new option to enable, and is just
in openssl(1) and not libcrypto, I'm disinclined to say "no" until
I see a PR (with documentation) that we can decide to leave for
post 1.1.1 or adopt.

-- 
Viktor.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
>We don't have to answer the question "how high" now.  I'm fully
prepared to have the use of iconv limited to platforms where we know
it's available

That then means that the pkcs12 program is not compatible in behavior across 
platforms.  This would be a first, for us.


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
In message  on Thu, 7 Jun 
2018 16:58:20 +0200, Andy Polyakov  said:

appro> One can argue that iconv was actually standardized, and in such
appro> case it would be appropriate to make it conditional on
appro> _POSIX_VERSION. [Though it doesn't seem to be part of pull
appro> request in question. Why not?] But as far as _POSIX_VERSION
appro> goes, we kind of know that some systems by *default* offer
appro> lower version, presumably in order to facilitate backward
appro> portability.

[about why not: because I was unsure how _POSIX_SOURCE is defined...
having seen too many places where the user (i.e. us) gets to define
that macro to get desired features.  I've read up since, so expect a
change that uses this macro]

appro> So that it would mean that we would have to explicitly rise the
appro> bar in some cases. Which ones? And how high? This brings us to
appro> following question. Is *this* actually right moment to
appro> introduce that kind of *multi-variable* problem? In other words
appro> the problem kind of has two sides: a) principal, to do or not
appro> to do; b) *when* would it be appropriate to start, is minor
appro> release right moment? Is b) part of the vote?

We don't have to answer the question "how high" now.  I'm fully
prepared to have the use of iconv limited to platforms where we know
it's available (for example, we - or well, *I* - know that VMS has the
iconv API in the C RTL, not even any need to link with any extra
library...  and we *know* it's available in glibc since version
2.something).  I'm fully prepared to have to deal with people saying
"hey, we have that too!" and having to edit config targets as we go.
I do not expect any support of iconv to cover more than what we can
test or get patches for, as with anything else.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
In message  on Thu, 7 Jun 
2018 11:56:00 -0400, Viktor Dukhovni  said:

openssl-users> 
openssl-users> 
openssl-users> > On Jun 7, 2018, at 11:19 AM, Richard Levitte 
 wrote:
openssl-users> > 
openssl-users> > Regarding general use of other libraries, please
openssl-users> > think carefully before voting, 'cause this *is*
openssl-users> > tricky. If you have a look, you will see that we
openssl-users> > *currently* depend on certain standard libraries,
openssl-users> > such as, for example, libdl. And perhaps we should
openssl-users> > also mention the pile of libraries used with
openssl-users> > windows.
openssl-users> > 
openssl-users> > In my mind, this makes that more general vote
openssl-users> > ridiculous, but the matter was brought up to me, and
openssl-users> > I wasn't going to ignore it, no matter what my
openssl-users> > personal feelings are.
openssl-users> 
openssl-users> My concern is not so much whether a dependency on libiconv in 
libcrypto
openssl-users> should be allowed, but rather wether we actually need it.  I 
rather
openssl-users> think that all codepage conversions should be the application's 
job.
openssl-users> 
openssl-users> Thus, it is OK for *apps* where we prompt for passwords to 
support
openssl-users> conversion to UTF-8, perhaps via libiconv.  So I see 
/usr/bin/openssl
openssl-users> linked against the iconv API (which is, for example, in libc on 
NetBSD
openssl-users> and FreeBSD, and does not require a separate library).  We 
probably
openssl-users> require libiconv for "openssl pkcs12" to work correctly, but the
openssl-users> dependency should IMHO be in apps not libcrypto.

Yup, and I did hear you in that other thread.  Your argument about
having OSSL_STORE be at liberty to *expect* UTF-8 without having to
check for it made sense to me, and I did move the check and possible
conversion to the application (i.e. 'openssl pkcs12').  That's what
the PR I pointed at is all about.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
In message  on Thu, 7 Jun 
2018 17:55:27 +0200, Andy Polyakov  said:

appro> > Regarding general use of other libraries, please think carefully 
before voting, 'cause this *is* tricky. If you have a look, you will see that 
we *currently* depend on certain standard libraries, such as, for example, 
libdl.
appro> 
appro> One has to recognize that each dependency has to be justified.

Yes, of course.  Caution is advisable, always.

appro> > And perhaps we should also mention the pile of libraries used with 
windows.
appro> 
appro> It's not about amount, but ubiquity and stability. Windows is bad
appro> example in the context, because it's rather "mono-cultural" environment.

Yes, you're right, and I didn't really mean to make a serious argument
about the amount, just pointing out that we already rely on standard /
system libraries.

appro> But *otherwise* thing is that we already *know* that those extra
appro> libraries work. Or at least know what to expect and how to deal with
appro> them on different platforms. They were effectively proven to work by
appro> lasting through several releases and years-long bug ironing. This *is*
appro> factor too. And that's what made me pose "is b) part of vote" in my last
appro> post.

I'll have you note that the PR that I'm pointing at still has an open
question about how to deal with the problem of some systems not
reliably support iconv.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
>Taking off on a bit of a tangent, how much justification did we go
through when adding pthreads as a dependency.

It's an optional, but enabled by default, dependency which is different.

We had a lot of discussion within what was then openssl-team.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Benjamin Kaduk
On Thu, Jun 07, 2018 at 05:55:27PM +0200, Andy Polyakov wrote:
> > Regarding general use of other libraries, please think carefully before 
> > voting, 'cause this *is* tricky. If you have a look, you will see that we 
> > *currently* depend on certain standard libraries, such as, for example, 
> > libdl.
> 
> One has to recognize that each dependency has to be justified. Sometimes
> you don't have a choice. For example if you want to talk network on
> Solaris, you have to link with -lsocket -lnsl. It's just part of the
> game. But in cases one has a choice, well, one has a choice to *make*.
> And key question is how do you anchor it. It's only natural to have as
> little dependencies as possible, so question is what would justify extra
> dependency.

Taking off on a bit of a tangent, how much justification did we go
through when adding pthreads as a dependency.  I have not been
following very much (Kurt would know more), but apparently in Debian
there are some issues regarding (statically linked?) applications
and libraries that use libcrypto but do not explicitly link with
-pthread.  "Issues" here being, IIRC, crashes at runtime.

-Ben
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Viktor Dukhovni



> On Jun 7, 2018, at 11:19 AM, Richard Levitte  wrote:
> 
> Regarding general use of other libraries, please think carefully before 
> voting, 'cause this *is* tricky. If you have a look, you will see that we 
> *currently* depend on certain standard libraries, such as, for example, 
> libdl. And perhaps we should also mention the pile of libraries used with 
> windows.
> 
> In my mind, this makes that more general vote ridiculous, but the matter was 
> brought up to me, and I wasn't going to ignore it, no matter what my personal 
> feelings are.

My concern is not so much whether a dependency on libiconv in libcrypto
should be allowed, but rather wether we actually need it.  I rather
think that all codepage conversions should be the application's job.

Thus, it is OK for *apps* where we prompt for passwords to support
conversion to UTF-8, perhaps via libiconv.  So I see /usr/bin/openssl
linked against the iconv API (which is, for example, in libc on NetBSD
and FreeBSD, and does not require a separate library).  We probably
require libiconv for "openssl pkcs12" to work correctly, but the
dependency should IMHO be in apps not libcrypto.

-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Andy Polyakov
> Regarding general use of other libraries, please think carefully before 
> voting, 'cause this *is* tricky. If you have a look, you will see that we 
> *currently* depend on certain standard libraries, such as, for example, libdl.

One has to recognize that each dependency has to be justified. Sometimes
you don't have a choice. For example if you want to talk network on
Solaris, you have to link with -lsocket -lnsl. It's just part of the
game. But in cases one has a choice, well, one has a choice to *make*.
And key question is how do you anchor it. It's only natural to have as
little dependencies as possible, so question is what would justify extra
dependency.

> And perhaps we should also mention the pile of libraries used with windows.

It's not about amount, but ubiquity and stability. Windows is bad
example in the context, because it's rather "mono-cultural" environment.
But *otherwise* thing is that we already *know* that those extra
libraries work. Or at least know what to expect and how to deal with
them on different platforms. They were effectively proven to work by
lasting through several releases and years-long bug ironing. This *is*
factor too. And that's what made me pose "is b) part of vote" in my last
post.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
Regarding general use of other libraries, please think carefully before voting, 
'cause this *is* tricky. If you have a look, you will see that we *currently* 
depend on certain standard libraries, such as, for example, libdl. And perhaps 
we should also mention the pile of libraries used with windows.

In my mind, this makes that more general vote ridiculous, but the matter was 
brought up to me, and I wasn't going to ignore it, no matter what my personal 
feelings are.

Cheers
Richard 

Richard Levitte  skrev: (7 juni 2018 13:54:11 CEST)
>Hi,
>
>This PR has been blocked, forcing a vote:
>
>https://github.com/openssl/openssl/pull/6392
>
>Background: we have been sloppy when producing PKCS#12 files, creating
>objects that aren't interoperable.  This can only happen with non-UTF8
>input methods, so this PR adds a higher level of control in the
>openssl application, so that it will do the best it can to make sure a
>pass phrase encoded with something other than UTF-8 gets correctly
>re-encoded, and failing that, try and make the user aware that they
>are about to create a non-interoperable object.  This triggered the
>use of the iconv API, and in the case of Mac OS/X, the use of the
>separate libiconv library.
>
>I'm going to make this into two votes, as both topics have come out
>because of this.
>
>1. A vote about general use of other libraries, limited to standard
>   system libraries, which may be platform dependent (I expect
>   libiconv on Mac OS/X to be such a library)
>
>2. A vote about the use of the iconv API
>
>Please discuss here, no in the vote threads.
>
>Cheers,
>Richard

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Andy Polyakov
> This PR has been blocked, forcing a vote:
> 
> https://github.com/openssl/openssl/pull/6392
> 
> Background: we have been sloppy when producing PKCS#12 files, creating
> objects that aren't interoperable.  This can only happen with non-UTF8
> input methods, so this PR adds a higher level of control in the
> openssl application, so that it will do the best it can to make sure a
> pass phrase encoded with something other than UTF-8 gets correctly
> re-encoded, and failing that, try and make the user aware that they
> are about to create a non-interoperable object.  This triggered the
> use of the iconv API, and in the case of Mac OS/X, the use of the
> separate libiconv library.

I find the reference to Mac OS X a bit misleading, because it suggests
that assessment was made on limited amount of data points. Basically on
how does it look on *contemporary* Linux/Unix platforms and Mac OS X.
But question runs deeper than that and should cover all platform that we
consider supporting. Which covers even ranges of older versions, in
sense that judging on latest version alone is hardly sufficient. For
example do we know *when* was libiconv introduced to Mac OS X? One can
naturally say that we are not obliged to care about *that* old versions,
but this is no excuse for not making thorougher assessment? I mean it's
only appropriate if we can answer the question how old does system have
to be for us to say "we don't care". And same question applies even to
other platforms, OpenBSD, FreeBSD, Android, Cygwin, Solaris, AIX, HP-UX,
DJGPP, Tru64, IRIX, ... One can argue that iconv was actually
standardized, and in such case it would be appropriate to make it
conditional on _POSIX_VERSION. [Though it doesn't seem to be part of
pull request in question. Why not?] But as far as _POSIX_VERSION goes,
we kind of know that some systems by *default* offer lower version,
presumably in order to facilitate backward portability. So that it would
mean that we would have to explicitly rise the bar in some cases. Which
ones? And how high? This brings us to following question. Is *this*
actually right moment to introduce that kind of *multi-variable*
problem? In other words the problem kind of has two sides: a) principal,
to do or not to do; b) *when* would it be appropriate to start, is minor
release right moment? Is b) part of the vote?
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Salz, Rich
I see you already started the votes.  No time for discussion?

I think OpenSSL should be a "fundamental" system library.  Perhaps the apps are 
different, but it should not require new libraries but could use them if 
available -- either at run-time or via config/build.

I think iconv in particular is a bad thing to require at this time, in a 1.1.1 
release.  It's not clear to me that it meets our API/ABI compatibility 
guarantee.  I also dislike iconv because of its size, the fact that it is a 
gross collection of hacks -- not its fault, it's the nature of charsets -- and 
that it is not universal.  This means that apps that "do the right thing" on 
some platforms, will FAIL to do so on opthers.

It is very very late in the release process to be adding a new dependency.

Finally, I believe that for this particular issue, we can add an API that 
enables applications to do the right thing, and we can add flags and warnings 
to the command-line that make it more clear when a user isn't doing the right 
thing (such as because they have existing files they need to read).

VOTE NO.

On 6/7/18, 8:04 AM, "Richard Levitte"  wrote:

Hi,

This PR has been blocked, forcing a vote:

https://github.com/openssl/openssl/pull/6392

Background: we have been sloppy when producing PKCS#12 files, creating
objects that aren't interoperable.  This can only happen with non-UTF8
input methods, so this PR adds a higher level of control in the
openssl application, so that it will do the best it can to make sure a
pass phrase encoded with something other than UTF-8 gets correctly
re-encoded, and failing that, try and make the user aware that they
are about to create a non-interoperable object.  This triggered the
use of the iconv API, and in the case of Mac OS/X, the use of the
separate libiconv library.

I'm going to make this into two votes, as both topics have come out
because of this.

1. A vote about general use of other libraries, limited to standard
   system libraries, which may be platform dependent (I expect
   libiconv on Mac OS/X to be such a library)

2. A vote about the use of the iconv API

Please discuss here, no in the vote threads.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

[openssl-project] To use or not use the iconv API, and to use or not use other libraries

2018-06-07 Thread Richard Levitte
Hi,

This PR has been blocked, forcing a vote:

https://github.com/openssl/openssl/pull/6392

Background: we have been sloppy when producing PKCS#12 files, creating
objects that aren't interoperable.  This can only happen with non-UTF8
input methods, so this PR adds a higher level of control in the
openssl application, so that it will do the best it can to make sure a
pass phrase encoded with something other than UTF-8 gets correctly
re-encoded, and failing that, try and make the user aware that they
are about to create a non-interoperable object.  This triggered the
use of the iconv API, and in the case of Mac OS/X, the use of the
separate libiconv library.

I'm going to make this into two votes, as both topics have come out
because of this.

1. A vote about general use of other libraries, limited to standard
   system libraries, which may be platform dependent (I expect
   libiconv on Mac OS/X to be such a library)

2. A vote about the use of the iconv API

Please discuss here, no in the vote threads.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project