konstantinb commented on code in PR #8426: URL: https://github.com/apache/hadoop/pull/8426#discussion_r3455709416
########## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ASharedThreadPoolDisabled.java: ########## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.fs.s3a; + +import org.assertj.core.api.Assertions; +import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Control case for {@link ITestS3ASharedThreadPoolEnabled}: with the shared + * pool disabled (the default), each S3AFileSystem's AWS SDK clients create their + * own sdk-ScheduledExecutor pool, so thread count grows with the number of + * instances. Kept a separate class so it runs in its own JVM (hadoop-aws uses + * reuseForks=false) and the static holders never memoize in the enabled state. Review Comment: Thank you @ajfabbri! Exactly the case I had in mind for spelling it out. Worth noting it's a per-process need, not per-thread: the holder is a JVM-global static, so these two want separate JVMs (the reuseForks axis), not just serialized threads — an ExecutionMode-style knob would need to drive fork allocation to cover it. FWIW I prototyped that side: each test launches a child JVM (ProcessBuilder, much like the router tests' JavaProcess) that reuses the same S3ATestUtils measurement, so the work runs with fresh statics and comes out reuseForks-immune [SharedPoolForkWorker.java](https://github.com/user-attachments/files/29225451/SharedPoolForkWorker.java) [ITestS3ASharedThreadPoolEnabledForked.java](https://github.com/user-attachments/files/29225480/ITestS3ASharedThreadPoolEnabledForked.java) [ITestS3ASharedThreadPoolDisabledForked.java](https://github.com/user-attachments/files/29225498/ITestS3ASharedThreadPoolDisabledForked.java) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
