lukecwik commented on code in PR #25300:
URL: https://github.com/apache/beam/pull/25300#discussion_r1096091240
##########
sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnboundedScheduledExecutorServiceTest.java:
##########
@@ -502,4 +509,45 @@ void wakeUpAndCheckTasks(UnboundedScheduledExecutorService
executorService) thro
}
Thread.sleep(100);
}
+
+ @Test
+ public void testThreadsAreAddedOnlyAsNeededWithContention() throws Exception
{
+ UnboundedScheduledExecutorService executorService = new
UnboundedScheduledExecutorService();
+ CountDownLatch done = new CountDownLatch(1);
+
+ ThreadPoolExecutor executor =
+ new ThreadPoolExecutor(100, 100, Long.MAX_VALUE, MILLISECONDS, new
SynchronousQueue<>());
+ // Schedule 1000 threads that are going to be scheduling work non-stop but
sequentially.
+ for (int i = 0; i < 100; ++i) {
+ executor.execute(
+ () -> {
+ // Periodically check if done.
+ while (done.getCount() == 1) {
+ for (int j = 0; j < 100; ++j) {
+ try {
+ executorService.submit(() -> {
+ try {
+ Thread.sleep(1);
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ }).get();
+ } catch (InterruptedException | ExecutionException e) {
+ // Ignore, happens on executor shutdown.
+ }
+ }
+ }
+ });
+ }
+
+ Thread.sleep(20 * 1000);
Review Comment:
I ran the test locally and it is flaking a lot (> 50%).
Without your change I see something like:
```
Feb 03, 2023 9:52:47 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 105 threads to execute at most 100 parallel tasks
Feb 03, 2023 9:52:48 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 107 threads to execute at most 100 parallel tasks
Feb 03, 2023 9:52:49 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 105 threads to execute at most 100 parallel tasks
Feb 03, 2023 9:52:50 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 105 threads to execute at most 100 parallel tasks
Feb 03, 2023 9:52:52 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 103 threads to execute at most 100 parallel tasks
```
With your change I see:
```
Feb 03, 2023 9:54:20 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 101 threads to execute at most 100 parallel tasks
Feb 03, 2023 9:54:21 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 100 threads to execute at most 100 parallel tasks
Feb 03, 2023 9:54:22 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 97 threads to execute at most 100 parallel tasks
Feb 03, 2023 9:54:24 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 101 threads to execute at most 100 parallel tasks
Feb 03, 2023 9:54:25 AM
org.apache.beam.sdk.util.UnboundedScheduledExecutorServiceTest
testThreadsAreAddedOnlyAsNeededWithContention
INFO: Created 101 threads to execute at most 100 parallel tasks
```
and it flaked 29 times out of 50.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
