[
https://issues.apache.org/jira/browse/SOLR-16720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709673#comment-17709673
]
Kevin Risden commented on SOLR-16720:
-------------------------------------
[~gerlowskija] is it possible this is causing the spikes in test
{code:java}
Build: https://jenkins.thetaphi.de/job/Solr-main-Linux/11253/
Java: 64bit/hotspot/jdk-20-rc -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.security.jwt.JWTAuthPluginIntegrationTest.testMetrics
Error Message:
java.lang.AssertionError: Expected metric minimums for prefix
SECURITY./authentication.: {failMissingCredentials=1, authenticated=4,
passThrough=4, failWrongCredentials=0, requests=9, errors=0}, but got:
{failMissingCredentials=1, authenticated=4, passThrough=2, totalTime=5816471,
failWrongCredentials=0, requestTimes=2208, requests=7, errors=0}(Possible cause
is delay in loading modified security.json; see SOLR-13464 for test work around)
Stack Trace:
java.lang.AssertionError: Expected metric minimums for prefix
SECURITY./authentication.: {failMissingCredentials=1, authenticated=4,
passThrough=4, failWrongCredentials=0, requests=9, errors=0}, but got:
{failMissingCredentials=1, authenticated=4, passThrough=2, totalTime=5816471,
failWrongCredentials=0, requestTimes=2208, requests=7, errors=0}(Possible cause
is delay in loading modified security.json; see SOLR-13464 for test work around)
at
__randomizedtesting.SeedInfo.seed([A22B6BB9DD7D5C81:5C39B91BC1772B32]:0)
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at
org.apache.solr.cloud.SolrCloudAuthTestCase.assertAuthMetricsMinimums(SolrCloudAuthTestCase.java:197)
at
org.apache.solr.cloud.SolrCloudAuthTestCase.assertAuthMetricsMinimums(SolrCloudAuthTestCase.java:132)
at
org.apache.solr.security.jwt.JWTAuthPluginIntegrationTest.testMetrics(JWTAuthPluginIntegrationTest.java:234)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:578)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at
org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
at
org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at
org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
at
org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at
org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at
org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at
org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at
org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at
org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at
org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
at java.base/java.lang.Thread.run(Thread.java:1623)
{code}
> PKI should decorate outgoing requests at "sending", not "enqueueing" time
> -------------------------------------------------------------------------
>
> Key: SOLR-16720
> URL: https://issues.apache.org/jira/browse/SOLR-16720
> Project: Solr
> Issue Type: Improvement
> Components: Authentication
> Affects Versions: 9.2
> Reporter: Jason Gerlowski
> Priority: Minor
> Attachments: SOLR-16720-reproduce.patch, reproduce.sh
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, PKIAuthenticationPlugin decorates intra-node requests using an
> 'onQueue' lifecycle hook, which is triggered when the request is enqueued for
> processing by the (asynchronous) Jetty http client.
> This works great on many systems. However on heavily loaded clusters the
> time between Jetty "queueing" the request and it actually being sent out can
> be non-negligible. If this gap becomes wide enough, the TTL encoded into the
> PKI auth header might have substantially or fully expired by the time the
> receiving node gets the request.
> We should experiment with moving PKI header decoration to the 'onBegin' hook
> instead, which fires much closer to the actual request-send time on heavily
> loaded servers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
