Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Salz, Rich
Tim and Viktor have convinced me that “it’s not worth it” is wrong.  Thanks, 
Richard, for testing 1.1.0 tests with 1.1.1 library. We do need to analyze the 
results and not say any failure means something 1.1.1 has to fix – it could be 
failing because of an assumption in the 1.1.0 tests.  Am I correct in saying 
that, so far, everything has been of that type?

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Tim Hudson
Where we are stating that ABI compatibility is in place we should be
testing it.
i.e. the older release binaries should be run against the current release
libraries - and that should be put into CI in my view.

Going the other direction isn't something I have thought we have ever
guaranteed (i.e. downgrading) - but programs that don't use new APIs (i.e.
for the ABI) should also work - that is pretty much the definition of an
ABI.
If we are unable to provide the forward ABI then we need to change the
version number of the release going forward. If weare unable to provide
backwards ABI then we need to think about how we are defining things and
make that clear.

And we need to be clear about what we mean by ABI - I don't think we have
written down a clear definition - and then have in CI the tests to match
what we are saying we do.

When it comes to TLSv1.3 it is highly desirable that an existing 1.1.0
application gets TLSv1.3 without API changes - i.e. ABI provides this.
There will be some things where there are problems where assumptions are
invalidated - but we should work to minimise those (and I think we have
actually done so).
But again we should be testing this in CI - if we want old apps to all get
TLSv1.3 for zero effort (other than a shared library upgrade) then we
should test that in CI. Hoping it works or assuming it works and "manually"
watching for changes that might invalidate that is rather error prone.

What Richard is doing really helps add to the information for informed
decisions ... the more information we have the better in my view.

Tim.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Viktor Dukhovni


> On Apr 15, 2018, at 12:59 PM, Salz, Rich  wrote:
> 
> Let me turn the question around because we'll never know "everything" just 
> works. Except for our tests, what programs work with 1.1.0 and *fail* to work 
> with 1.1.1?  Any? For various reasons that Viktor and I have detailed, *our 
> tests* do not count.

I would not go as far as that.  Our tests do count, and should be paid 
attention to, but we need to be careful about interpreting the results.  
Sometimes the answer is to tune the test to make it portable to later library 
versions.  So far, we've not seen issues that warrant a library version bump, 
but testing this is a good idea, and Richard is doing good work.

-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Viktor Dukhovni


> On Apr 15, 2018, at 12:55 PM, Salz, Rich  wrote:
> 
> Do our 1.1.0 tests work when linked against the 1.1.1 library?

Our tests don't, but Richard (valiantly I must say) went to the trouble
of doing just that.  And found some tests that failed, ...

> Even then, there might be some failures because some of those tests probably 
> say "pick any protocol" and they were written at a time when 1.3 was not 
> available so might explicitly test, for example, that "any protocol" meant 
> "got 1.2"

in particular this type of failure.

> It would be interesting to test 1.1.0 against the 1.1.1 library, and then 
> analyze the failures and see which, if any, indicate bugs in the 1.1.1 
> compatibility.

This is what Richard was doing, and I commend his efforts.

> Again, to repeat myself, we have datapoints that 1.1.0 programs can use 1.1.1 
> library with no problems. We do not have any datapoints that typical 1.1.0 
> programs fail when using 1.1.1 library. 

I think that tests of this sort are valuable, and should in some cases make the 
test's assumptions more explicit, so that they will also pass with later 
libraries.  Then we can focus on any "real" issues that come up, and decide 
whether they are bugs, significant incompatibilities or "artificial" issues not 
substantive for "real" applications.

-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Salz, Rich
> I believe we were led into the current situation, because our tests don't 
completely work *going backwards.*  Do the 1.1.0 tests basically work *going 
forwards* ?

>It is unclear what you mean by forwards and backwards, but some 1.1.0
tests failed when using a 1.1.1 library.  However, the tests that I
read about failing were testing artificial expectations that are only
appropriate for the same library as the tests.  The tests can be fixed
to make their expectations more explicit (by e.g. setting the max protocol
version to the largest supported by the corresponding library).
  
Good point.  I meant our 1.1.1 tests don't completely work when linked with 
1.1.0 library.  I am not surprised about that as I am sure there are all sorts 
of hidden assumptions in the 1.1.1 tests.  Now it seems to turn out that there 
are only a couple of assumptions, and that maybe we can fix them.  As I said 
initially, I don't see that as worth any effort, but others are free to 
disagree and have.

Do our 1.1.0 tests work when linked against the 1.1.1 library?  Even then, 
there might be some failures because some of those tests probably say "pick any 
protocol" and they were written at a time when 1.3 was not available so might 
explicitly test, for example, that "any protocol" meant "got 1.2"  It would be 
interesting to test 1.1.0 against the 1.1.1 library, and then analyze the 
failures and see which, if any, indicate bugs in the 1.1.1 compatibility.

Again, to repeat myself, we have datapoints that 1.1.0 programs can use 1.1.1 
library with no problems. We do not have any datapoints that typical 1.1.0 
programs fail when using 1.1.1 library. 

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Viktor Dukhovni


> On Apr 15, 2018, at 2:24 AM, Bernd Edlinger  wrote:
> 
> One possible example of application failure that I am aware of is #5743:
> A certificate that is incompatible with TLS1.3 but works with TLS1.2.
> Admittedly that I did come up with that scenario only because I saw
> a possible issue per code inspection.

[ Repeating in part my response to Richar's mesage also in this thread ]

This is a bug that needs to be fixed, the point format for TLS does not
have any provenance over X.509.  There's no such thing as a certificate
not compatible with TLS 1.3 (that is compatible with TLS 1.2).

-- 
Viktor.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


[openssl-project] Proto over ciphers or ciphers over proto? (was: The problem of (implicit) relinking and changed behaviour)

2018-04-15 Thread Richard Levitte
In message 

 on Sun, 15 Apr 2018 06:24:48 +, Bernd Edlinger  
said:

bernd.edlinger> One possible example of application failure that I am aware of 
is #5743:
bernd.edlinger> A certificate that is incompatible with TLS1.3 but works with 
TLS1.2.
bernd.edlinger> Admittedly that I did come up with that scenario only because I 
saw
bernd.edlinger> a possible issue per code inspection.

This touches an issue that's already mentioned in Matt's blog, and I
gotta ask how the protocols so be presented for negotiation are chosen
(yes, I know, I could dive into the code...  and I will unless there's
a quick answer).  Does libssl just pick the max version chosen (within
the range that we support unless the application has narrowed it
down), or does it also look at other facts, such as chosen server or
client certs to see what protocol version range would actually work
with those collected facts?  #5743 seems to say that libssl doesn't
look at such facts, and can end up in the absurd situation that things
stop working because it selected TLSv1.3 over TLSv1.2 when the latter
couldn't possibly work right, while TLSv1.2 does.

I can't really say what's right or wrong in this case, this really is
a philosophical question more than anything else.  Is it all right to
just pick a proto version that cannot work and then virtually flip it
to the unsuspecting application that wasn't prepared with better data
(such as a cert that's also valid in TLSv1.3) or is that essentially
wrong, even though easier to deal with in code?  Is that what libssl
is doing, or does it have more of a "look at all the facts" approach
before choosing the proto range to negotiate with the other end?

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Richard Levitte
In message  on Sun, 15 Apr 
2018 13:27:15 +0200, Andy Polyakov  said:

appro> To summarize, failing tests in 110 should be revisited to see if they
appro> are actually representative, before one can consider as drastic measures
appro> as #5945.

At this point, I agree.  Viktor's look at several applications and
Kurt's report of successful did convince me that I might by crying
wolf a bit much.  I've started looking a bit deeper at the failing
tests, for exactly the reasons you mention.

Now, there was a point brought on by a couple of issues mentioned,
I'll take that in a separate email.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Andy Polyakov
> 2. Make TLSv1.2 the absolutely maximum TLS version available for
>programs linked with libssl 1.1.0.  This is what's done in this PR:
>https://github.com/openssl/openssl/pull/5945
>This makes sense insofar that it's safe, it works within the known
>parameters for the library these programs were built for.

Here is a thing. If we assume that 110/test/sslapitest.c is
*representative* example of an 1.1.0 application, then it's not at all
given that #5945 would actually solve anything. Indeed, observing the
first failing test, recompiled 110/test/sslapitest.c would still fail,
because it makes assumption that session id is available right after
negotiation, something that doesn't hold true for TLS 1.3. This gives
you either a) no 1.1.0 application goes safe, recompiled[!] or not; b)
110/test/sslapitest.c is not representative example. Well, as far as
first failing test goes, I'd say it's b), which means that it should be
adjusted in one way or another or failing check omitted. [It's b),
because it's unnatural to put session id you've just got back to id
cache, the test is ultimately synthetic.] This naturally doesn't answer
all the questions, but it does show that above mentioned premise is
somewhat flawed. I mean "programs were built for [1.1.0]" would still
work if recompiled with #5945.

>It also makes sense if we view TLSv1.3 as new functionality, and
>new functionality is usually only available to those who
>explicitely build their programs for the new library version.
>TLSv1.3 is unusual in this sense because it's at least it great
>part "under the hood", just no 100% transparently so.

In such case it should rather be #ifdef OPENSSL_ENABLE_TLS1_3 instead of
#ifndef OPENSSL_DISABLE_TLS1_3. And we don't want that.

To summarize, failing tests in 110 should be revisited to see if they
are actually representative, before one can consider as drastic measures
as #5945.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Kurt Roeckx
On Sun, Apr 15, 2018 at 07:38:48AM +0200, Richard Levitte wrote:
> In message  on Sat, 14 Apr 
> 2018 21:13:47 +, "Salz, Rich"  said:
> 
> rsalz> We have *no* data points, except our tests, that anything fails to 
> work.
> rsalz> In fact, we are increasingly collecting data that shows everything is 
> just fine.
> 
> Errr, are we?  Please inform me, because I cannot remember having seen
> tests that specifically targets the case of programs built with 1.1.0
> that get implicitly relinked with 1.1.1 libraries (that's what you
> call "going forward", isn't it?), or data collection for that matter.
> I may have missed something, but I am interested.

In Debian we've done a rebuild test of all source packages that link
to 1.1.0 to build against pre2 instead, and as far as I know only 1
package was found to fail because of that: #5637

This is of course slightly different than just upgrading the
library since it can pick up new header files, but I think this is
close enough. It also only covers those packages that have a test
suite.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project


Re: [openssl-project] The problem of (implicit) relinking and changed behaviour

2018-04-15 Thread Bernd Edlinger
On 04/15/18 07:53, Viktor Dukhovni wrote:
> 
> 
>> On Apr 15, 2018, at 1:38 AM, Richard Levitte  wrote:
>>
>> Errr, are we?  Please inform me, because I cannot remember having seen
>> tests that specifically targets the case of programs built with 1.1.0
>> that get implicitly relinked with 1.1.1 libraries (that's what you
>> call "going forward", isn't it?), or data collection for that matter.
>> I may have missed something, but I am interested.
> 
> It think it is most prudent to not fall into the trap of debating this
> particular side-issue.  I commend your initiative of running the 1.1.0
> tests against the 1.1.1 libraries, that's fantastic.  And I further
> commend attention to the failure cases.  Thank you.
> 
> With that out of the way, it seems to me that apart from some fixes in
> the test framework, and tests that did not expect protocol versions
> higher than TLS 1.2, no *interesting* issues have turned up.
> 
> If such issue did or will turn up let's fix them, but there should not
> be fundamental obstacles to an ABI-compatible 1.1.1 library with the
> same SONAME as its 1.1.0 predecessor.  The new library may negotiate
> TLS 1.3 which 1.1.0 did not, but I don't see that as an incompatibility
> that requires an SONAME version bump.
> 
> Which is not to say I could not be convinced otherwise, but at present
> I don't see a need for the bump, or for work-arounds to limit the
> negotiated protocols for code compiled against 1.1.0 that happens to
> run against 1.1.1.
> 
> Let's stay alert, but not overreact to minor issues we can resolve.
> 

One possible example of application failure that I am aware of is #5743:
A certificate that is incompatible with TLS1.3 but works with TLS1.2.
Admittedly that I did come up with that scenario only because I saw
a possible issue per code inspection.


Bernd.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project