hfukada commented on PR #15049: URL: https://github.com/apache/druid/pull/15049#issuecomment-1875570802
A couple of build failures seem unrelated to my changes? Static Checks CI / intellij-inspections (pull_request): https://github.com/apache/druid/actions/runs/7388106247/job/20112155029?pr=15049 I did not touch any of these files and I'm confused why these are failing. ``` Done. Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/msq/ITMultiStageQuery.java:116 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/msq/ITMultiStageQuery.java:52 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/msq/ITMultiStageQueryWorkerFaultTolerance.java:65 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/msq/ITKeyStatisticsSketchMergeMode.java:125 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/msq/ITKeyStatisticsSketchMergeMode.java:55 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/msq/ITKeyStatisticsSketchMergeMode.java:196 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/AzureTestUtil.java:122 -- The declared exception <code>URISyntaxException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/AzureTestUtil.java:75 -- The declared exception <code>URISyntaxException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/AzureTestUtil.java:75 -- The declared exception <code>InvalidKeyException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/AzureTestUtil.java:85 -- The declared exception <code>URISyntaxException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/AzureTestUtil.java:109 -- The declared exception <code>URISyntaxException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/GcsTestUtil.java:108 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/GcsTestUtil.java:92 -- The declared exception <code>IOException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/GcsTestUtil.java:75 -- The declared exception <code>GeneralSecurityException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/GcsTestUtil.java:75 -- The declared exception <code>IOException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/GcsTestUtil.java:102 -- The declared exception <code>IOException</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/config/DruidTestRunner.java:78 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/config/DruidTestRunner.java:60 -- The declared exception <code>Throwable</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/indexer/ITLocalInputSourceAllFormatSchemalessTest.java:94 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/indexer/ITLocalInputSourceAllFormatSchemalessTest.java:43 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/indexer/ITLocalInputSourceAllFormatSchemalessTest.java:80 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/indexer/ITLocalInputSourceAllFormatSchemalessTest.java:108 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/indexer/ITLocalInputSourceAllFormatSchemalessTest.java:122 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/indexer/AbstractITBatchIndexTest.java:185 -- The declared exception <code>Exception</code> is never thrown Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/config/Initializer.java:352 -- Call to <code>asList</code> with only one argument #loc Error: integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/utils/DruidClusterAdminClient.java:250 -- Suspicious call to 'List.contains' Error: integration-tests-ex/cases/pom.xml:21 -- Failed to read artifact descriptor for org.yaml:snakeyaml:jar:1.21 #loc Error: Process completed with exit code 1. ``` Unit & Integration tests CI / standard-its / integration-query-tests-middleManager (centralized-datasource-schema) / centralized-datasource-schema integration test (Compile=jdk8, Run=jdk8, Indexer=middleManager, Mysql=com.mysql.jdbc.Driver) (pull_request): [Details](https://github.com/apache/druid/actions/runs/7388106321/job/20113023928?pr=15049) ``` [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running TestSuite 2024-01-03T04:12:16,541 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 6.2.5.Final 2024-01-03T04:12:18,605 INFO [main] org.apache.druid.storage.google.GoogleStorageDruidModule - Getting jackson modules... 2024-01-03T04:12:18,706 WARN [main] org.apache.druid.initialization.Log4jShutterDownerModule - Shutdown callback registry expected class [org.apache.druid.common.config.Log4jShutdown] found [org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry]. Skipping shutdown registry 2024-01-03T04:12:19,577 INFO [main] org.apache.druid.storage.google.GoogleStorageDruidModule - Configuring GoogleStorageDruidModule... 2024-01-03T04:12:19,954 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2024-01-03T04:12:21,313 INFO [main] org.apache.druid.server.metrics.MetricsModule - Loaded 2 monitors: org.apache.druid.curator.DruidConnectionStateListener, org.apache.druid.server.initialization.jetty.JettyServerModule$JettyMonitor 2024-01-03T04:12:21,536 INFO [main] org.apache.druid.https.SSLContextProvider - Creating SslContext for https client using config [SSLClientConfig{protocol='null', trustStoreType='null', trustStorePath='client_tls/truststore.jks', trustStoreAlgorithm='null', keyStorePath='client_tls/client.jks', keyStoreType='null', certAlias='druid', keyManagerFactoryAlgorithm='null', validateHostnames='null'}] 2024-01-03T04:12:22,825 INFO [main] org.apache.druid.https.SSLContextProvider - Creating SslContext for https client using config [SSLClientConfig{protocol='null', trustStoreType='null', trustStorePath='client_tls/truststore.jks', trustStoreAlgorithm='null', keyStorePath='client_tls/client.jks', keyStoreType='null', certAlias='druid', keyManagerFactoryAlgorithm='null', validateHostnames='null'}] 2024-01-03T04:12:23,452 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Trying attempt[0/240]... 2024-01-03T04:12:23,635 WARN [main] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[http://127.0.0.1:8081/] was returned multiple times? 2024-01-03T04:12:23,636 ERROR [main] org.apache.druid.testing.utils.DruidClusterAdminClient - Error while waiting for [http://127.0.0.1:8081/] to be ready java.util.concurrent.ExecutionException: java.nio.channels.ClosedChannelException at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:592) ~[guava-32.0.1-jre.jar:?] at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:551) ~[guava-32.0.1-jre.jar:?] at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:111) ~[guava-32.0.1-jre.jar:?] at org.apache.druid.testing.utils.DruidClusterAdminClient.lambda$waitUntilInstanceReady$1(DruidClusterAdminClient.java:268) ~[druid-integration-tests-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.apache.druid.testing.utils.ITRetryUtil.retryUntil(ITRetryUtil.java:61) ~[druid-integration-tests-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.apache.druid.testing.utils.ITRetryUtil.retryUntilTrue(ITRetryUtil.java:39) ~[druid-integration-tests-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.apache.druid.testing.utils.DruidClusterAdminClient.waitUntilInstanceReady(DruidClusterAdminClient.java:262) ~[druid-integration-tests-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.apache.druid.testing.utils.DruidClusterAdminClient.waitUntilCoordinatorReady(DruidClusterAdminClient.java:124) ~[druid-integration-tests-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.apache.druid.testing.utils.SuiteListener.onStart(SuiteListener.java:41) ~[druid-integration-tests-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.testng.SuiteRunner.invokeListeners(SuiteRunner.java:223) ~[testng-7.3.0.jar:?] at org.testng.SuiteRunner.run(SuiteRunner.java:284) ~[testng-7.3.0.jar:?] at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53) ~[testng-7.3.0.jar:?] at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96) ~[testng-7.3.0.jar:?] at org.testng.TestNG.runSuitesSequentially(TestNG.java:1218) ~[testng-7.3.0.jar:?] at org.testng.TestNG.runSuitesLocally(TestNG.java:1140) ~[testng-7.3.0.jar:?] at org.testng.TestNG.runSuites(TestNG.java:1069) ~[testng-7.3.0.jar:?] at org.testng.TestNG.run(TestNG.java:1037) ~[testng-7.3.0.jar:?] at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:283) ~[surefire-testng-2.22.2.jar:2.22.2] at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75) ~[surefire-testng-2.22.2.jar:2.22.2] at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:120) ~[surefire-testng-2.22.2.jar:2.22.2] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) ~[surefire-booter-2.22.2.jar:2.22.2] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) ~[surefire-booter-2.22.2.jar:2.22.2] at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) ~[surefire-booter-2.22.2.jar:2.22.2] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) ~[surefire-booter-2.22.2.jar:2.22.2] Caused by: java.nio.channels.ClosedChannelException at org.jboss.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:433) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromUserCode(AbstractNioWorker.java:128) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:84) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.Channels.write(Channels.java:725) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.handler.codec.http.HttpClientCodec.handleDownstream(HttpClientCodec.java:97) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.Channels.write(Channels.java:704) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.Channels.write(Channels.java:671) ~[netty-3.10.6.Final.jar:?] at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:347) ~[netty-3.10.6.Final.jar:?] at org.apache.druid.java.util.http.client.NettyHttpClient.go(NettyHttpClient.java:396) ~[druid-processing-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.apache.druid.java.util.http.client.CredentialedHttpClient.go(CredentialedHttpClient.java:48) ~[druid-processing-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.apache.druid.java.util.http.client.AbstractHttpClient.go(AbstractHttpClient.java:33) ~[druid-processing-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] at org.apache.druid.testing.utils.DruidClusterAdminClient.lambda$waitUntilInstanceReady$1(DruidClusterAdminClient.java:265) ~[druid-integration-tests-29.0.0-SNAPSHOT.jar:29.0.0-SNAPSHOT] ... 20 more 2024-01-03T04:12:23,680 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Attempt[0/240] did not pass: Task Waiting for instance to be ready: [http://127.0.0.1:8081/] still not complete. Next retry in 5000 ms 2024-01-03T04:12:28,680 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Trying attempt[1/240]... 2024-01-03T04:12:28,683 ERROR [main] org.apache.druid.testing.utils.DruidClusterAdminClient - Channel Closed 2024-01-03T04:12:28,683 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Attempt[1/240] did not pass: Task Waiting for instance to be ready: [http://127.0.0.1:8081/] still not complete. Next retry in 5000 ms 2024-01-03T04:12:28,684 WARN [HttpClient-Netty-Worker-20] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[http://127.0.0.1:8081/] was returned multiple times? 2024-01-03T04:12:33,683 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Trying attempt[2/240]... 2024-01-03T04:12:33,689 ERROR [main] org.apache.druid.testing.utils.DruidClusterAdminClient - Channel Closed 2024-01-03T04:12:33,689 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Attempt[2/240] did not pass: Task Waiting for instance to be ready: [http://127.0.0.1:8081/] still not complete. Next retry in 5000 ms 2024-01-03T04:12:33,690 WARN [HttpClient-Netty-Worker-21] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[http://127.0.0.1:8081/] was returned multiple times? 2024-01-03T04:12:38,690 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Trying attempt[3/240]... 2024-01-03T04:12:38,692 ERROR [main] org.apache.druid.testing.utils.DruidClusterAdminClient - Channel Closed 2024-01-03T04:12:38,692 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Attempt[3/240] did not pass: Task Waiting for instance to be ready: [http://127.0.0.1:8081/] still not complete. Next retry in 5000 ms 2024-01-03T04:12:38,693 WARN [HttpClient-Netty-Worker-22] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[http://127.0.0.1:8081/] was returned multiple times? 2024-01-03T04:12:43,693 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Trying attempt[4/240]... 2024-01-03T04:12:43,695 WARN [main] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[http://127.0.0.1:8081/] was returned multiple times? 2024-01-03T04:12:43,697 ERROR [main] org.apache.druid.testing.utils.DruidClusterAdminClient - Channel disconnected 2024-01-03T04:12:43,697 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Attempt[4/240] did not pass: Task Waiting for instance to be ready: [http://127.0.0.1:8081/] still not complete. Next retry in 5000 ms 2024-01-03T04:12:48,698 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Trying attempt[5/240]... 2024-01-03T04:12:48,700 WARN [main] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[http://127.0.0.1:8081/] was returned multiple times? 2024-01-03T04:12:48,700 ERROR [main] org.apache.druid.testing.utils.DruidClusterAdminClient - Channel disconnected 2024-01-03T04:12:48,700 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Attempt[5/240] did not pass: Task Waiting for instance to be ready: [http://127.0.0.1:8081/] still not complete. Next retry in 5000 ms 2024-01-03T04:12:53,701 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Trying attempt[6/240]... 2024-01-03T04:12:53,702 ERROR [main] org.apache.druid.testing.utils.DruidClusterAdminClient - Channel Closed 2024-01-03T04:12:53,702 WARN [HttpClient-Netty-Worker-25] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[http://127.0.0.1:8081/] was returned multiple times? 2024-01-03T04:12:53,703 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Attempt[6/240] did not pass: Task Waiting for instance to be ready: [http://127.0.0.1:8081/] still not complete. Next retry in 5000 ms 2024-01-03T04:12:58,703 INFO [main] org.apache.druid.testing.utils.ITRetryUtil - Trying attempt[7/240]... ``` Also does not look to be my fault, and I cannot retry it. Code scanning results / CodeQL Failing after 4s — 1 new alert including 1 high severity security vulnerability [Details](https://github.com/apache/druid/pull/15049/checks?check_run_id=20112261203) I don't know how other PRs pass this given that this `limit(position + numBytes)` is copied. I don't actually think it'll even throw an overflow exception on this line either. it's just limit. I pulled this example from the other already-merged sketches as inspiration. The buffer is readonly, and the java protobuf library implements various memory safeguards when using `parseFrom(bytes)`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
