Mrxiashu opened a new issue #11312:
URL: https://github.com/apache/druid/issues/11312
The message "Failed to publish segments to DB" is displayed when the
Kafka-index is used by Druid. Check whether there is a code bug.
### Affected Version
0.21.0
### Description
When the Kafka-index mode is used to ingest data, the task remains in the
running state after the taskDuration expires, and the log contains "Failed to
publish segments to DB."
- Cluster size
2 master 8U32G
1 query 8U32G
2 historical 16U32G
3 middlemanager 16U32G
- Configurations in use
common.runtime.properties:
druid.metadata.storage.type=mysql
druid.metadata.storage.connector.connectURI=jdbc:mysql://XXXX:3306/datacompassdb?passwordCharacterEncoding=utf8&useSSL=false&autoReconnect=true&serverTimezone=UTC&rewriteBatchedStatements=true
druid.metadata.storage.connector.user=XXXX
druid.metadata.storage.connector.password=XXXX
druid.metadata.storage.connector.dbcp.maxTotal=64
druid.access.jdbc.enforceAllowedProperties=false
druid.access.jdbc.allowedProperties=["useSSL", "requireSSL", "ssl",
"sslmode", "autoReconnect", "serverTimezone", "passwordCharacterEncoding",
"rewriteBatchedStatements"]
- Nearby log messages
index-kafka-task-xxxxx.log:
2021-05-28T01:29:15,471 WARN [[index_kafka_xxxxxxx]-publish]
org.apache.druid.indexing.common.actions.RemoteTaskActionClient - Will try
again in [PT8.717S].
coordinator-overlord.log:
2021-05-28T01:35:43,938 ERROR [qtp1274308620-129]
org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator - Exception
inserting segments:
[xxxxxx_2021-05-28T01:27:00.000Z_2021-05-28T01:28:00.000Z_2021-05-28T01:23:47.980Z,
xxxxxxxx_2021-05-28T01:29:00.000Z_2021-05-28T01:30:00.000Z_2021-05-28T01:25:48.028Z,
xxxxxxxx_2021-05-28T01:03:00.000Z_2021-05-28T01:04:00.000Z_2021-05-28T00:59:48.023Z_3,
xxxxxxxx_2021-05-28T01:04:00.000Z_2021-05-28T01:05:00.000Z_2021-05-28T01:00:48.042Z_3,
xxxxxxxx_2021-05-28T01:10:00.000Z_2021-05-28T01:11:00.000Z_2021-05-28T01:06:48.019Z_1,
xxxxxxxx_2021-05-28T01:01:00.000Z_2021-05-28T01:02:00.000Z_2021-05-28T00:57:48.026Z_3,
xxxxxxxx_2021-05-28T01:15:00.000Z_2021-05-28T01:16:00.000Z_2021-05-28T01:11:48.013Z_1,
xxxxxxxx_2021-05-28T01:09:00.000Z_2021-05-28T01:10:00.000Z_2021-05-28T01:05:48.025Z_2,
xxxxxxxx_2021-05-28T01:24:00.000Z_2021-05-28T01:25:00.000Z_2021-05-28T01:21:59.770Z,
xxxxxxxx_2021-05-28T01:21:00.000Z_2021-05-28T01:22:00.000Z_2021-05-28T01:1
7:48.010Z_1,
xxxxxxxx_2021-05-28T01:08:00.000Z_2021-05-28T01:09:00.000Z_2021-05-28T01:04:48.010Z_2,
xxxxxxxx_2021-05-28T01:12:00.000Z_2021-05-28T01:13:00.000Z_2021-05-28T01:08:48.012Z_2,
xxxxxxxx_2021-05-28T01:00:00.000Z_2021-05-28T01:01:00.000Z_2021-05-28T00:56:49.231Z_3,
xxxxxxxx_2021-05-28T00:59:00.000Z_2021-05-28T01:00:00.000Z_2021-05-28T00:55:48.027Z_4,
xxxxxxxx_2021-05-28T01:19:00.000Z_2021-05-28T01:20:00.000Z_2021-05-28T01:17:10.277Z_1,
xxxxxxxx_2021-05-28T01:17:00.000Z_2021-05-28T01:18:00.000Z_2021-05-28T01:13:48.030Z_2,
xxxxxxxx_2021-05-28T01:16:00.000Z_2021-05-28T01:17:00.000Z_2021-05-28T01:12:47.998Z_1,
xxxxxxxx_2021-05-28T01:23:00.000Z_2021-05-28T01:24:00.000Z_2021-05-28T01:21:46.589Z,
xxxxxxxx_2021-05-28T01:28:00.000Z_2021-05-28T01:29:00.000Z_2021-05-28T01:24:47.999Z,
xxxxxxxx_2021-05-28T01:25:00.000Z_2021-05-28T01:26:00.000Z_2021-05-28T01:22:11.824Z,
xxxxxxxx_2021-05-28T01:14:00.000Z_2021-05-28T01:15:00.000Z_2021-05-28T01:10:48.015Z_1,
xxxxxxxx_2021-05-28T01:18:00.000Z
_2021-05-28T01:19:00.000Z_2021-05-28T01:14:48.035Z_2,
xxxxxxxx_2021-05-28T01:11:00.000Z_2021-05-28T01:12:00.000Z_2021-05-28T01:07:48.019Z_1,
xxxxxxxx_2021-05-28T01:20:00.000Z_2021-05-28T01:21:00.000Z_2021-05-28T01:17:21.856Z_1,
xxxxxxxx_2021-05-28T01:26:00.000Z_2021-05-28T01:27:00.000Z_2021-05-28T01:22:48.021Z,
xxxxxxxx_2021-05-28T01:22:00.000Z_2021-05-28T01:23:00.000Z_2021-05-28T01:21:34.799Z,
xxxxxxxx_2021-05-28T01:02:00.000Z_2021-05-28T01:03:00.000Z_2021-05-28T00:58:48.032Z_2,
xxxxxxxx_2021-05-28T01:07:00.000Z_2021-05-28T01:08:00.000Z_2021-05-28T01:03:48.035Z_2,
xxxxxxxx_2021-05-28T01:13:00.000Z_2021-05-28T01:14:00.000Z_2021-05-28T01:09:48.009Z_2,
xxxxxxxx_2021-05-28T01:06:00.000Z_2021-05-28T01:07:00.000Z_2021-05-28T01:02:48.026Z_4,
xxxxxxxx_2021-05-28T00:58:00.000Z_2021-05-28T00:59:00.000Z_2021-05-28T00:54:48.031Z_4,
xxxxxxxx_2021-05-28T01:05:00.000Z_2021-05-28T01:06:00.000Z_2021-05-28T01:01:48.029Z_4]
2021-05-28T01:35:43,946 WARN [qtp1274308620-129]
org.apache.druid.indexing.overlord.http.OverlordResource - Failed to perform
task action
java.lang.RuntimeException:
org.skife.jdbi.v2.exceptions.CallbackFailedException:
org.apache.druid.java.util.common.ISE: Failed to publish segments to DB:
[xxxxxxxx_2021-05-28T01:27:00.000Z_2021-05-28T01:28:00.000Z_2021-05-28T01:23:47.980Z,
xxxxxxxx_2021-05-28T01:29:00.000Z_2021-05-28T01:30:00.000Z_2021-05-28T01:25:48.028Z,
xxxxxxxx_2021-05-28T01:03:00.000Z_2021-05-28T01:04:00.000Z_2021-05-28T00:59:48.023Z_3,
xxxxxxxx_2021-05-28T01:04:00.000Z_2021-05-28T01:05:00.000Z_2021-05-28T01:00:48.042Z_3,
xxxxxxxx_2021-05-28T01:10:00.000Z_2021-05-28T01:11:00.000Z_2021-05-28T01:06:48.019Z_1,
xxxxxxxx_2021-05-28T01:01:00.000Z_2021-05-28T01:02:00.000Z_2021-05-28T00:57:48.026Z_3,
xxxxxxxx_2021-05-28T01:15:00.000Z_2021-05-28T01:16:00.000Z_2021-05-28T01:11:48.013Z_1,
xxxxxxxx_2021-05-28T01:09:00.000Z_2021-05-28T01:10:00.000Z_2021-05-28T01:05:48.025Z_2,
xxxxxxxx_2021-05-28T01:24:00.000Z_2021-05-28T01:25:00.000Z_2021-05-28T01:21:59.770Z,
xxxxxxxx_2021-05-28T01:21:00.000Z_2021-05-28T01:22:00.000Z_202
1-05-28T01:17:48.010Z_1,
xxxxxxxx_2021-05-28T01:08:00.000Z_2021-05-28T01:09:00.000Z_2021-05-28T01:04:48.010Z_2,
xxxxxxxx_2021-05-28T01:12:00.000Z_2021-05-28T01:13:00.000Z_2021-05-28T01:08:48.012Z_2,
xxxxxxxx_2021-05-28T01:00:00.000Z_2021-05-28T01:01:00.000Z_2021-05-28T00:56:49.231Z_3,
xxxxxxxx_2021-05-28T00:59:00.000Z_2021-05-28T01:00:00.000Z_2021-05-28T00:55:48.027Z_4,
xxxxxxxx_2021-05-28T01:19:00.000Z_2021-05-28T01:20:00.000Z_2021-05-28T01:17:10.277Z_1,
xxxxxxxx_2021-05-28T01:17:00.000Z_2021-05-28T01:18:00.000Z_2021-05-28T01:13:48.030Z_2,
xxxxxxxx_2021-05-28T01:16:00.000Z_2021-05-28T01:17:00.000Z_2021-05-28T01:12:47.998Z_1,
xxxxxxxx_2021-05-28T01:23:00.000Z_2021-05-28T01:24:00.000Z_2021-05-28T01:21:46.589Z,
xxxxxxxx_2021-05-28T01:28:00.000Z_2021-05-28T01:29:00.000Z_2021-05-28T01:24:47.999Z,
xxxxxxxx_2021-05-28T01:25:00.000Z_2021-05-28T01:26:00.000Z_2021-05-28T01:22:11.824Z,
xxxxxxxx_2021-05-28T01:14:00.000Z_2021-05-28T01:15:00.000Z_2021-05-28T01:10:48.015Z_1,
xxxxxxxx_2021-05-28T0
1:18:00.000Z_2021-05-28T01:19:00.000Z_2021-05-28T01:14:48.035Z_2,
xxxxxxxx_2021-05-28T01:11:00.000Z_2021-05-28T01:12:00.000Z_2021-05-28T01:07:48.019Z_1,
xxxxxxxx_2021-05-28T01:20:00.000Z_2021-05-28T01:21:00.000Z_2021-05-28T01:17:21.856Z_1,
xxxxxxxx_2021-05-28T01:26:00.000Z_2021-05-28T01:27:00.000Z_2021-05-28T01:22:48.021Z,
xxxxxxxx_2021-05-28T01:22:00.000Z_2021-05-28T01:23:00.000Z_2021-05-28T01:21:34.799Z,
xxxxxxxx_2021-05-28T01:02:00.000Z_2021-05-28T01:03:00.000Z_2021-05-28T00:58:48.032Z_2,
xxxxxxxx_2021-05-28T01:07:00.000Z_2021-05-28T01:08:00.000Z_2021-05-28T01:03:48.035Z_2,
xxxxxxxx_2021-05-28T01:13:00.000Z_2021-05-28T01:14:00.000Z_2021-05-28T01:09:48.009Z_2,
xxxxxxxx_2021-05-28T01:06:00.000Z_2021-05-28T01:07:00.000Z_2021-05-28T01:02:48.026Z_4,
xxxxxxxx_2021-05-28T00:58:00.000Z_2021-05-28T00:59:00.000Z_2021-05-28T00:54:48.031Z_4,
xxxxxxxx_2021-05-28T01:05:00.000Z_2021-05-28T01:06:00.000Z_2021-05-28T01:01:48.029Z_4]
at
org.apache.druid.indexing.common.actions.SegmentTransactionalInsertAction.perform(SegmentTransactionalInsertAction.java:211)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.common.actions.SegmentTransactionalInsertAction.perform(SegmentTransactionalInsertAction.java:55)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.common.actions.LocalTaskActionClient.submit(LocalTaskActionClient.java:74)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.overlord.http.OverlordResource$4.apply(OverlordResource.java:490)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.overlord.http.OverlordResource$4.apply(OverlordResource.java:479)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.overlord.http.OverlordResource.asLeaderWith(OverlordResource.java:855)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.overlord.http.OverlordResource.doAction(OverlordResource.java:476)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at sun.reflect.GeneratedMethodAccessor131.invoke(Unknown Source)
~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_262]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_262]
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
~[jersey-server-1.19.3.jar:1.19.3]
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
~[jersey-servlet-1.19.3.jar:1.19.3]
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
~[jersey-servlet-1.19.3.jar:1.19.3]
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
~[jersey-servlet-1.19.3.jar:1.19.3]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
~[javax.servlet-api-3.1.0.jar:3.1.0]
at
com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
~[guice-servlet-4.1.0.jar:?]
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
~[guice-servlet-4.1.0.jar:?]
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
~[guice-servlet-4.1.0.jar:?]
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
~[guice-servlet-4.1.0.jar:?]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
~[guice-servlet-4.1.0.jar:?]
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120)
~[guice-servlet-4.1.0.jar:?]
at
com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
~[guice-servlet-4.1.0.jar:?]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.apache.druid.server.http.RedirectFilter.doFilter(RedirectFilter.java:73)
~[druid-server-0.21.0.jar:0.21.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.doFilter(PreResponseAuthorizationCheckFilter.java:82)
~[druid-server-0.21.0.jar:0.21.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.apache.druid.server.security.AllowHttpMethodsResourceFilter.doFilter(AllowHttpMethodsResourceFilter.java:78)
~[druid-server-0.21.0.jar:0.21.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.apache.druid.server.security.AllowOptionsResourceFilter.doFilter(AllowOptionsResourceFilter.java:75)
~[druid-server-0.21.0.jar:0.21.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.apache.druid.server.security.AllowAllAuthenticator$1.doFilter(AllowAllAuthenticator.java:84)
~[druid-server-0.21.0.jar:0.21.0]
at
org.apache.druid.server.security.AuthenticationWrappingFilter.doFilter(AuthenticationWrappingFilter.java:59)
~[druid-server-0.21.0.jar:0.21.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.apache.druid.server.security.SecuritySanityCheckFilter.doFilter(SecuritySanityCheckFilter.java:77)
~[druid-server-0.21.0.jar:0.21.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
~[jetty-servlet-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at org.eclipse.jetty.server.Server.handle(Server.java:516)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
~[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:279)
[jetty-server-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
[jetty-io-9.4.39.v20210325.jar:9.4.39.v20210325]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
[jetty-io-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
[jetty-io-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
[jetty-io-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
[jetty-io-9.4.39.v20210325.jar:9.4.39.v20210325]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
[jetty-io-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
[jetty-io-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
[jetty-util-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
[jetty-util-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
[jetty-util-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
[jetty-util-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:383)
[jetty-util-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)
[jetty-util-9.4.39.v20210325.jar:9.4.39.v20210325]
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)
[jetty-util-9.4.39.v20210325.jar:9.4.39.v20210325]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]
Caused by: org.skife.jdbi.v2.exceptions.CallbackFailedException:
org.apache.druid.java.util.common.ISE: Failed to publish segments to DB:
[xxxxxxxx_2021-05-28T01:27:00.000Z_2021-05-28T01:28:00.000Z_2021-05-28T01:23:47.980Z,
xxxxxxxx_2021-05-28T01:29:00.000Z_2021-05-28T01:30:00.000Z_2021-05-28T01:25:48.028Z,
xxxxxxxx_2021-05-28T01:03:00.000Z_2021-05-28T01:04:00.000Z_2021-05-28T00:59:48.023Z_3,
xxxxxxxx_2021-05-28T01:04:00.000Z_2021-05-28T01:05:00.000Z_2021-05-28T01:00:48.042Z_3,
xxxxxxxx_2021-05-28T01:10:00.000Z_2021-05-28T01:11:00.000Z_2021-05-28T01:06:48.019Z_1,
xxxxxxxx_2021-05-28T01:01:00.000Z_2021-05-28T01:02:00.000Z_2021-05-28T00:57:48.026Z_3,
xxxxxxxx_2021-05-28T01:15:00.000Z_2021-05-28T01:16:00.000Z_2021-05-28T01:11:48.013Z_1,
xxxxxxxx_2021-05-28T01:09:00.000Z_2021-05-28T01:10:00.000Z_2021-05-28T01:05:48.025Z_2,
xxxxxxxx_2021-05-28T01:24:00.000Z_2021-05-28T01:25:00.000Z_2021-05-28T01:21:59.770Z,
xxxxxxxx_2021-05-28T01:21:00.000Z_2021-05-28T01:22:00.000Z_2021-05-28T01:17:48.
010Z_1,
xxxxxxxx_2021-05-28T01:08:00.000Z_2021-05-28T01:09:00.000Z_2021-05-28T01:04:48.010Z_2,
xxxxxxxx_2021-05-28T01:12:00.000Z_2021-05-28T01:13:00.000Z_2021-05-28T01:08:48.012Z_2,
xxxxxxxx_2021-05-28T01:00:00.000Z_2021-05-28T01:01:00.000Z_2021-05-28T00:56:49.231Z_3,
xxxxxxxx_2021-05-28T00:59:00.000Z_2021-05-28T01:00:00.000Z_2021-05-28T00:55:48.027Z_4,
xxxxxxxx_2021-05-28T01:19:00.000Z_2021-05-28T01:20:00.000Z_2021-05-28T01:17:10.277Z_1,
xxxxxxxx_2021-05-28T01:17:00.000Z_2021-05-28T01:18:00.000Z_2021-05-28T01:13:48.030Z_2,
xxxxxxxx_2021-05-28T01:16:00.000Z_2021-05-28T01:17:00.000Z_2021-05-28T01:12:47.998Z_1,
xxxxxxxx_2021-05-28T01:23:00.000Z_2021-05-28T01:24:00.000Z_2021-05-28T01:21:46.589Z,
xxxxxxxx_2021-05-28T01:28:00.000Z_2021-05-28T01:29:00.000Z_2021-05-28T01:24:47.999Z,
xxxxxxxx_2021-05-28T01:25:00.000Z_2021-05-28T01:26:00.000Z_2021-05-28T01:22:11.824Z,
xxxxxxxx_2021-05-28T01:14:00.000Z_2021-05-28T01:15:00.000Z_2021-05-28T01:10:48.015Z_1,
xxxxxxxx_2021-05-28T01:18:00.000Z_2021
-05-28T01:19:00.000Z_2021-05-28T01:14:48.035Z_2,
xxxxxxxx_2021-05-28T01:11:00.000Z_2021-05-28T01:12:00.000Z_2021-05-28T01:07:48.019Z_1,
xxxxxxxx_2021-05-28T01:20:00.000Z_2021-05-28T01:21:00.000Z_2021-05-28T01:17:21.856Z_1,
xxxxxxxx_2021-05-28T01:26:00.000Z_2021-05-28T01:27:00.000Z_2021-05-28T01:22:48.021Z,
xxxxxxxx_2021-05-28T01:22:00.000Z_2021-05-28T01:23:00.000Z_2021-05-28T01:21:34.799Z,
xxxxxxxx_2021-05-28T01:02:00.000Z_2021-05-28T01:03:00.000Z_2021-05-28T00:58:48.032Z_2,
xxxxxxxx_2021-05-28T01:07:00.000Z_2021-05-28T01:08:00.000Z_2021-05-28T01:03:48.035Z_2,
xxxxxxxx_2021-05-28T01:13:00.000Z_2021-05-28T01:14:00.000Z_2021-05-28T01:09:48.009Z_2,
xxxxxxxx_2021-05-28T01:06:00.000Z_2021-05-28T01:07:00.000Z_2021-05-28T01:02:48.026Z_4,
xxxxxxxx_2021-05-28T00:58:00.000Z_2021-05-28T00:59:00.000Z_2021-05-28T00:54:48.031Z_4,
xxxxxxxx_2021-05-28T01:05:00.000Z_2021-05-28T01:06:00.000Z_2021-05-28T01:01:48.029Z_4]
at org.skife.jdbi.v2.DBI.withHandle(DBI.java:284)
~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.DBI.inTransaction(DBI.java:353)
~[jdbi-2.63.1.jar:2.63.1]
at
org.apache.druid.metadata.SQLMetadataConnector.lambda$retryTransaction$1(SQLMetadataConnector.java:154)
~[druid-server-0.21.0.jar:0.21.0]
at
org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:87)
~[druid-core-0.21.0.jar:0.21.0]
at
org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:115)
~[druid-core-0.21.0.jar:0.21.0]
at
org.apache.druid.metadata.SQLMetadataConnector.retryTransaction(SQLMetadataConnector.java:154)
~[druid-server-0.21.0.jar:0.21.0]
at
org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator.announceHistoricalSegments(IndexerSQLMetadataStorageCoordinator.java:373)
~[druid-server-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.common.actions.SegmentTransactionalInsertAction.lambda$perform$0(SegmentTransactionalInsertAction.java:195)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.overlord.CriticalAction.perform(CriticalAction.java:51)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.overlord.TaskLockbox.doInCriticalSection(TaskLockbox.java:559)
~[druid-indexing-service-0.21.0.jar:0.21.0]
at
org.apache.druid.indexing.common.actions.SegmentTransactionalInsertAction.perform(SegmentTransactionalInsertAction.java:190)
~[druid-indexing-service-0.21.0.jar:0.21.0]
... 87 more
- Debugging
After debugging the code, it is found that in the
announceHistoricalSegmentBatch method, when multiple records exist in
preparedBatch and preparedBatch.execute() is executed, the returned value is –2
instead of 1 when the data is written successfully.
In addition, when the code goes to the PreparedStatement method (in
mysql-connector-java version 5.1.48)
if (numBatchedArgs > 1) {
long updCount = updateCountRunningTotal > 0L ? -2L : 0L;
....
The part that determines the write state does not have a return value of 1.
The logic for determining the write status in the
announceHistoricalSegmentBatch function is modified as follows:
final boolean succeeded =
Arrays.stream(affectedRows).allMatch(eachAffectedRows -> eachAffectedRows == 1
|| eachAffectedRows == -2);
The data is written successfully, and the task scrolls normally.
Is this a code bug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
