Have a fix up here: https://github.com/apache/solr/pull/653 (Though I can't
reproduce locally anyways)

Note, I also fixed the test to not need a SolrCore, but it's not the reason
why the test was failing. Might split that into a separate PR.

On Fri, Feb 18, 2022 at 11:42 AM Kevin Risden <[email protected]> wrote:

> It looks like these errors were introduced by SOLR-15950:
>
> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/jenkins/workspace/Solr-main-Linux/solr/core/build/resources/test/solr/filestore"
> "write")
>
> Kevin Risden
>
>
> On Fri, Feb 18, 2022 at 6:59 AM Policeman Jenkins Server <
> [email protected]> wrote:
>
>> Build: https://jenkins.thetaphi.de/job/Solr-main-Linux/3046/
>> Java: 64bit/jdk-17.0.2 -XX:+UseCompressedOops -XX:+UseParallelGC
>>
>> 3 tests failed.
>> FAILED:  org.apache.solr.rest.TestManagedResourceStorage.classMethod
>>
>> Error Message:
>> java.security.AccessControlException: access denied
>> ("java.io.FilePermission"
>> "/home/jenkins/workspace/Solr-main-Linux/solr/core/build/resources/test/solr/filestore"
>> "write")
>>
>> Stack Trace:
>> java.security.AccessControlException: access denied
>> ("java.io.FilePermission"
>> "/home/jenkins/workspace/Solr-main-Linux/solr/core/build/resources/test/solr/filestore"
>> "write")
>>         at __randomizedtesting.SeedInfo.seed([C621B3A80C0AD024]:0)
>>         at
>> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:485)
>>         at
>> java.base/java.security.AccessController.checkPermission(AccessController.java:1068)
>>         at
>> java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:416)
>>         at
>> java.base/java.lang.SecurityManager.checkWrite(SecurityManager.java:847)
>>         at java.base/sun.nio.fs.UnixPath.checkWrite(UnixPath.java:787)
>>         at
>> java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
>>         at java.base/java.nio.file.Files.createDirectory(Files.java:700)
>>         at
>> java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
>>         at java.base/java.nio.file.Files.createDirectories(Files.java:753)
>>         at
>> org.apache.solr.filestore.DistribPackageStore.getPackageStoreDirPath(DistribPackageStore.java:584)
>>         at
>> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:567)
>>         at
>> org.apache.solr.filestore.DistribPackageStore.<init>(DistribPackageStore.java:83)
>>         at
>> org.apache.solr.filestore.PackageStoreAPI.<init>(PackageStoreAPI.java:76)
>>         at org.apache.solr.core.CoreContainer.load(CoreContainer.java:725)
>>         at org.apache.solr.util.TestHarness.<init>(TestHarness.java:184)
>>         at org.apache.solr.util.TestHarness.<init>(TestHarness.java:156)
>>         at org.apache.solr.util.TestHarness.<init>(TestHarness.java:162)
>>         at org.apache.solr.util.TestHarness.<init>(TestHarness.java:112)
>>         at
>> org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:813)
>>         at
>> org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:803)
>>         at
>> org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:650)
>>         at
>> org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:639)
>>         at
>> org.apache.solr.cloud.AbstractZkTestCase.azt_beforeClass(AbstractZkTestCase.java:68)
>>         at
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>>         at
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
>>         at
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.base/java.lang.reflect.Method.invoke(Method.java:568)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:882)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
>>         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>         at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>         at
>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>>         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>         at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at
>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>         at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>         at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at
>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>         at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>         at
>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>         at
>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>         at
>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>         at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
>>         at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
>>         at java.base/java.lang.Thread.run(Thread.java:833)
>>
>>
>> FAILED:  org.apache.solr.rest.TestManagedResourceStorage.classMethod
>>
>> Error Message:
>> com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from
>> SUITE scope at org.apache.solr.rest.TestManagedResourceStorage:
>>    1) Thread[id=145, name=zkConnectionManagerCallback-96-thread-1,
>> state=WAITING, group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.park(LockSupport.java:341)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:506)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1623)
>>         at [email protected]
>> /java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    2) Thread[id=139, name=h2sc-91-thread-1, state=RUNNABLE,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
>>         at [email protected]/sun.nio.ch
>> .EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.select(SelectorImpl.java:146)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.nioSelect(ManagedSelector.java:183)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.select(ManagedSelector.java:190)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.select(ManagedSelector.java:606)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>>         at 
>> app//org.eclipse.jetty.io.ManagedSelector$$Lambda$299/0x000000080102e258.run(Unknown
>> Source)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:235)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$300/0x000000080102ed08.run(Unknown
>> Source)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    3) Thread[id=136, name=httpShardExecutor-89-thread-1, state=RUNNABLE,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
>>         at [email protected]/sun.nio.ch
>> .EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.select(SelectorImpl.java:146)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.nioSelect(ManagedSelector.java:183)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.select(ManagedSelector.java:190)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.select(ManagedSelector.java:606)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>>         at 
>> app//org.eclipse.jetty.io.ManagedSelector$$Lambda$299/0x000000080102e258.run(Unknown
>> Source)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:235)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$300/0x000000080102ed08.run(Unknown
>> Source)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    4) Thread[id=146,
>> name=OverseerStateUpdate-72481201937383427-127.0.0.1:8983_-n_0000000000,
>> state=TIMED_WAITING, group=Overseer state updater.]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
>>         at
>> app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:477)
>>         at
>> app//org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:294)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    5) Thread[id=144,
>> name=SUITE-TestManagedResourceStorage-seed#[C621B3A80C0AD024]-worker-EventThread,
>> state=WAITING, group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.park(LockSupport.java:341)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:506)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1623)
>>         at [email protected]
>> /java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:551)
>>    6) Thread[id=147,
>> name=OverseerCollectionConfigSetProcessor-72481201937383427-127.0.0.1:8983_-n_0000000000,
>> state=TIMED_WAITING, group=Overseer collection creation process.]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
>>         at
>> app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:477)
>>         at
>> app//org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:271)
>>         at
>> app//org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:258)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    7) Thread[id=143,
>> name=SUITE-TestManagedResourceStorage-seed#[C621B3A80C0AD024]-worker-SendThread(
>> 127.0.0.1:39935), state=TIMED_WAITING,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/java.lang.Thread.sleep(Native Method)
>>         at
>> app//org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:214)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1380)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1321)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1299)
>>    8) Thread[id=148, name=zkCallback-95-thread-1, state=TIMED_WAITING,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
>>         at [email protected]
>> /java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:401)
>>         at [email protected]
>> /java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>
>> Stack Trace:
>> com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from
>> SUITE scope at org.apache.solr.rest.TestManagedResourceStorage:
>>    1) Thread[id=145, name=zkConnectionManagerCallback-96-thread-1,
>> state=WAITING, group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.park(LockSupport.java:341)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:506)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1623)
>>         at [email protected]
>> /java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    2) Thread[id=139, name=h2sc-91-thread-1, state=RUNNABLE,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
>>         at [email protected]/sun.nio.ch
>> .EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.select(SelectorImpl.java:146)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.nioSelect(ManagedSelector.java:183)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.select(ManagedSelector.java:190)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.select(ManagedSelector.java:606)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>>         at 
>> app//org.eclipse.jetty.io.ManagedSelector$$Lambda$299/0x000000080102e258.run(Unknown
>> Source)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:235)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$300/0x000000080102ed08.run(Unknown
>> Source)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    3) Thread[id=136, name=httpShardExecutor-89-thread-1, state=RUNNABLE,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
>>         at [email protected]/sun.nio.ch
>> .EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.select(SelectorImpl.java:146)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.nioSelect(ManagedSelector.java:183)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.select(ManagedSelector.java:190)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.select(ManagedSelector.java:606)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>>         at 
>> app//org.eclipse.jetty.io.ManagedSelector$$Lambda$299/0x000000080102e258.run(Unknown
>> Source)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:235)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$300/0x000000080102ed08.run(Unknown
>> Source)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    4) Thread[id=146,
>> name=OverseerStateUpdate-72481201937383427-127.0.0.1:8983_-n_0000000000,
>> state=TIMED_WAITING, group=Overseer state updater.]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
>>         at
>> app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:477)
>>         at
>> app//org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:294)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    5) Thread[id=144,
>> name=SUITE-TestManagedResourceStorage-seed#[C621B3A80C0AD024]-worker-EventThread,
>> state=WAITING, group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.park(LockSupport.java:341)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:506)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1623)
>>         at [email protected]
>> /java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:551)
>>    6) Thread[id=147,
>> name=OverseerCollectionConfigSetProcessor-72481201937383427-127.0.0.1:8983_-n_0000000000,
>> state=TIMED_WAITING, group=Overseer collection creation process.]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
>>         at
>> app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:477)
>>         at
>> app//org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:271)
>>         at
>> app//org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:258)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    7) Thread[id=143,
>> name=SUITE-TestManagedResourceStorage-seed#[C621B3A80C0AD024]-worker-SendThread(
>> 127.0.0.1:39935), state=TIMED_WAITING,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/java.lang.Thread.sleep(Native Method)
>>         at
>> app//org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:214)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1380)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1321)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1299)
>>    8) Thread[id=148, name=zkCallback-95-thread-1, state=TIMED_WAITING,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
>>         at [email protected]
>> /java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:401)
>>         at [email protected]
>> /java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>         at __randomizedtesting.SeedInfo.seed([C621B3A80C0AD024]:0)
>>
>>
>> FAILED:  org.apache.solr.rest.TestManagedResourceStorage.classMethod
>>
>> Error Message:
>> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still
>> zombie threads that couldn't be terminated:
>>    1) Thread[id=145, name=zkConnectionManagerCallback-96-thread-1,
>> state=WAITING, group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.park(LockSupport.java:341)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:506)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1623)
>>         at [email protected]
>> /java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    2) Thread[id=139, name=h2sc-91-thread-1, state=RUNNABLE,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
>>         at [email protected]/sun.nio.ch
>> .EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.select(SelectorImpl.java:146)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.nioSelect(ManagedSelector.java:183)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.select(ManagedSelector.java:190)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.select(ManagedSelector.java:606)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>>         at 
>> app//org.eclipse.jetty.io.ManagedSelector$$Lambda$299/0x000000080102e258.run(Unknown
>> Source)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:235)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$300/0x000000080102ed08.run(Unknown
>> Source)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    3) Thread[id=136, name=httpShardExecutor-89-thread-1, state=RUNNABLE,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
>>         at [email protected]/sun.nio.ch
>> .EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.select(SelectorImpl.java:146)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.nioSelect(ManagedSelector.java:183)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.select(ManagedSelector.java:190)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.select(ManagedSelector.java:606)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>>         at 
>> app//org.eclipse.jetty.io.ManagedSelector$$Lambda$299/0x000000080102e258.run(Unknown
>> Source)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:235)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$300/0x000000080102ed08.run(Unknown
>> Source)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    4) Thread[id=143,
>> name=SUITE-TestManagedResourceStorage-seed#[C621B3A80C0AD024]-worker-SendThread(
>> 127.0.0.1:39935), state=TIMED_WAITING,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/java.lang.Thread.sleep(Native Method)
>>         at
>> app//org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:214)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1380)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1321)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1299)
>>    5) Thread[id=148, name=zkCallback-95-thread-1, state=TIMED_WAITING,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
>>         at [email protected]
>> /java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:401)
>>         at [email protected]
>> /java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>
>> Stack Trace:
>> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still
>> zombie threads that couldn't be terminated:
>>    1) Thread[id=145, name=zkConnectionManagerCallback-96-thread-1,
>> state=WAITING, group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.park(LockSupport.java:341)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:506)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463)
>>         at [email protected]
>> /java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434)
>>         at [email protected]
>> /java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1623)
>>         at [email protected]
>> /java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    2) Thread[id=139, name=h2sc-91-thread-1, state=RUNNABLE,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
>>         at [email protected]/sun.nio.ch
>> .EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.select(SelectorImpl.java:146)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.nioSelect(ManagedSelector.java:183)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.select(ManagedSelector.java:190)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.select(ManagedSelector.java:606)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>>         at 
>> app//org.eclipse.jetty.io.ManagedSelector$$Lambda$299/0x000000080102e258.run(Unknown
>> Source)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:235)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$300/0x000000080102ed08.run(Unknown
>> Source)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    3) Thread[id=136, name=httpShardExecutor-89-thread-1, state=RUNNABLE,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
>>         at [email protected]/sun.nio.ch
>> .EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
>>         at [email protected]/sun.nio.ch
>> .SelectorImpl.select(SelectorImpl.java:146)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.nioSelect(ManagedSelector.java:183)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector.select(ManagedSelector.java:190)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.select(ManagedSelector.java:606)
>>         at app//org.eclipse.jetty.io
>> .ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>>         at
>> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>>         at 
>> app//org.eclipse.jetty.io.ManagedSelector$$Lambda$299/0x000000080102e258.run(Unknown
>> Source)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:235)
>>         at
>> app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$300/0x000000080102ed08.run(Unknown
>> Source)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>    4) Thread[id=143,
>> name=SUITE-TestManagedResourceStorage-seed#[C621B3A80C0AD024]-worker-SendThread(
>> 127.0.0.1:39935), state=TIMED_WAITING,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/java.lang.Thread.sleep(Native Method)
>>         at
>> app//org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:214)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1380)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1321)
>>         at
>> app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1299)
>>    5) Thread[id=148, name=zkCallback-95-thread-1, state=TIMED_WAITING,
>> group=TGRP-TestManagedResourceStorage]
>>         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
>>         at [email protected]
>> /java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
>>         at [email protected]
>> /java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:401)
>>         at [email protected]
>> /java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
>>         at [email protected]
>> /java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>>         at [email protected]/java.lang.Thread.run(Thread.java:833)
>>         at __randomizedtesting.SeedInfo.seed([C621B3A80C0AD024]:0)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>
>

Reply via email to