[
https://issues.apache.org/jira/browse/SOLR-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-5776:
---------------------------
Attachment: SOLR-5776.patch
Having spent the past week+ looking into bugs in the SSL test setup, and
getting a lot more familiar with the various Java SSL APIs, I started
rethinking about this overall issue. Knowing what i know now, I went back and
re-read every comment, and re-reviewed the commits, here's a few things i
noticed...
* NullSecureRandom was only ever used in the SSLContext registered with
_clients_ ...
** Jetty instances were configured to use an SslContextFactory using the
*string* "SHA1PRNG"
*** SHA1PRNG is suppose to be faster then NativePRNG on linux, but (IIUC) still
requires true entropy for seeding
*** It is however possible to initialize jetty with an explicit SSLContext
_object_ such that it should never even attempt to initialize it's own
SecureRandom. So just like with the client code path, we can provide the
explicit (Null)SecureRandom object for jetty to use.
* NullSecureRandom implemented _some_ methods as No-Op but...
** it had no constructor, meaning it implicitly use {{SecureRandom()}}
*** This means the Platform default RNG Provider was loaded, and it may have
it's own entropy blocking code on init
*** if we define our own NullSecureRandom constructor that explicitly calls
{{super(SecureRandomSpi,Provider)}} we should be able to bypass the loading of
the system default (and any init it entails) and use our own
{{NullSecureRandomSpi}}
** it overrode {{setSeed(byte[] seed)}} but not {{setSeed(long seed)}}
*** if any caller code (in jetty, or in the JVM) tried using {{setSeed(long}}
that also could have resulted in using some entropy blocking method in the
default (super class) RNG provider
With this in mind, I set out to see if I could revive the old patch
w/improvements, which i'm now attaching.
I was never personally able to reproduce any of the "Tests take so long they
crash because of lack of entroy" type problems that seemd to plague the jetty
machines, but I did do some rough timings on my laptop that seem to suggest
that this patch definitely reduces the overhead/time of the tests...
* w/o patch...{code}
ant jar && cd solr/core && ant test -Dtests.seed=DEADBEEF
...
Total time: 33 minutes 15 seconds
{code}
* w/ patch...{code}
ant jar && cd solr/core && ant test -Dtests.seed=DEADBEEF
...
Total time: 20 minutes 39 seconds
{code}
* w/patch + override SSL randomization to always use SSL+clientAuth...{code}
// log.info("Randomized ssl ({}) and clientAuth ({})", trySsl,
trySslClientAuth);
// return new SSLTestConfig(trySsl, trySslClientAuth);
log.info("nocommit: forcing SSL on test that does not have @SuppressSSL");
return new SSLTestConfig(true, true);
{code}{code}
ant jar && cd solr/core && ant test -Dtests.seed=DEADBEEF
...
Total time: 34 minutes 39 seconds
{code}
...admitedly there were two (reproducible) OOMs in that last case (when forcing
SSL+clientAuth) that i'm still looking into, but I suspect these may just be be
because the responses are very large, and the SSL overhead pushes them over the
edge - I've definitely seen the TestDistributedSearch OOM from jenkins not too
long ago when it randomly selected SSL+clientAuth...
{noformat}
[junit4] Tests with failures [seed: DEADBEEF]:
[junit4] -
org.apache.solr.handler.component.TestDistributedStatsComponentCardinality.test
[junit4] - org.apache.solr.TestDistributedSearch.test
{noformat}
...in any case, this patch definitely seems like it helps in terms of test
performance. Even if we don't want to increase therandomization factors, this
sems like it would help.
> Look at speeding up using SSL with tests.
> -----------------------------------------
>
> Key: SOLR-5776
> URL: https://issues.apache.org/jira/browse/SOLR-5776
> Project: Solr
> Issue Type: Test
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.9, master
>
> Attachments: SOLR-5776.patch, SOLR-5776.patch, SOLR-5776.patch
>
>
> We have to disable SSL on a bunch of tests now because it appears to sometime
> be ridiculously slow - especially in slow envs (I never see timeouts on my
> machine).
> I was talking to Robert about this, and he mentioned that there might be some
> settings we could change to speed it up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
