JRobTS opened a new issue, #19036:
URL: https://github.com/apache/druid/issues/19036
The Unified Console classic-batch-data-loader fails when reindexing data
from existing Druid segments that contain a quantilesDoublesSketch fails with
error: `Failed to sample data: Unhandled type: COMPLEX`
### Affected Version
33.0.0
### Description
In the Connect step of the classic-batch-data-loader, with source type
`druid`, select a Druid datasource that contains a quantilesDoublesSketch and
an appropriate interval. Upon clicking Apply we see: `Failed to sample data:
Unhandled type: COMPLEX`.
The coordinator log contains a stack trace:
```
2026-02-19T17:24:31,503 INFO [qtp615774072-197]
org.apache.druid.segment.loading.LocalDataSegmentPuller - Copied 791578758
bytes from
[/data/druid/deep-storage/segments/datasource/2025-01-01T00:00:00.000Z_2025-01-02T00:00:00.000Z/2025-01-02T00:15:10.922Z/0/index]
to
[/data/druid/apache-druid-33.0.0/var/tmp/druid11891466104252998799/datasource/2025-01-01T00:00:00.000Z_2025-01-02T00:00:00.000Z/2025-01-02T00:15:10.922Z/0]
2026-02-19T17:24:31,510 WARN [qtp615774072-197]
org.apache.druid.indexing.input.DruidSegmentInputEntity - Could not clean
temporary segment file:
var/tmp/druid11891466104252998799/datasource/2025-01-01T00:00:00.000Z_2025-01-02T00:00:00.000Z/2025-01-02T00:15:10.922Z/0
2026-02-19T17:21:17,014 ERROR [qtp615774072-197]
org.apache.druid.indexing.overlord.sampler.SamplerExceptionMapper - Failed to
sample data: Unhandled type: COMPLEX
org.apache.druid.indexing.overlord.sampler.SamplerException: Failed to
sample data: Unhandled type: COMPLEX
at
org.apache.druid.indexing.overlord.sampler.InputSourceSampler.sample(InputSourceSampler.java:290)
~[druid-indexing-service-33.0.0.jar:33.0.0]
at
org.apache.druid.indexing.overlord.sampler.IndexTaskSamplerSpec.sample(IndexTaskSamplerSpec.java:90)
~[druid-indexing-service-33.0.0.jar:33.0.0]
at
org.apache.druid.indexing.overlord.sampler.SamplerResource.post(SamplerResource.java:84)
~[druid-indexing-service-33.0.0.jar:33.0.0]
at jdk.internal.reflect.GeneratedMethodAccessor718.invoke(Unknown
Source) ~[?:?]
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:569) ~[?:?]
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
~[jersey-server-1.19.4.jar:1.19.4]
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
~[jersey-servlet-1.19.4.jar:1.19.4]
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
~[jersey-servlet-1.19.4.jar:1.19.4]
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
~[jersey-servlet-1.19.4.jar:1.19.4]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
~[javax.servlet-api-3.1.0.jar:3.1.0]
at
com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290)
~[guice-servlet-5.1.0.jar:?]
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280)
~[guice-servlet-5.1.0.jar:?]
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184)
~[guice-servlet-5.1.0.jar:?]
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:89)
~[guice-servlet-5.1.0.jar:?]
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
~[guice-servlet-5.1.0.jar:?]
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121)
~[guice-servlet-5.1.0.jar:?]
at
com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
~[guice-servlet-5.1.0.jar:?]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.apache.druid.server.http.RedirectFilter.doFilter(RedirectFilter.java:73)
~[druid-server-33.0.0.jar:33.0.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.doFilter(PreResponseAuthorizationCheckFilter.java:84)
~[druid-server-33.0.0.jar:33.0.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.apache.druid.server.initialization.jetty.StandardResponseHeaderFilterHolder$StandardResponseHeaderFilter.doFilter(StandardResponseHeaderFilterHolder.java:164)
~[druid-server-33.0.0.jar:33.0.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.apache.druid.server.security.AllowHttpMethodsResourceFilter.doFilter(AllowHttpMethodsResourceFilter.java:78)
~[druid-server-33.0.0.jar:33.0.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.apache.druid.server.security.AllowOptionsResourceFilter.doFilter(AllowOptionsResourceFilter.java:74)
~[druid-server-33.0.0.jar:33.0.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.apache.druid.server.security.AllowAllAuthenticator$1.doFilter(AllowAllAuthenticator.java:84)
~[druid-server-33.0.0.jar:33.0.0]
at
org.apache.druid.server.security.AuthenticationWrappingFilter.doFilter(AuthenticationWrappingFilter.java:59)
~[druid-server-33.0.0.jar:33.0.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.apache.druid.server.security.SecuritySanityCheckFilter.doFilter(SecuritySanityCheckFilter.java:77)
~[druid-server-33.0.0.jar:33.0.0]
at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at org.eclipse.jetty.server.Server.handle(Server.java:516)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
[jetty-io-9.4.57.v20241219.jar:9.4.57.v20241219]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
[jetty-io-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
[jetty-io-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
[jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
[jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
[jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
[jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
[jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
[jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: org.apache.druid.java.util.common.IAE: Unhandled type: COMPLEX
at
org.apache.druid.segment.AutoTypeColumnIndexer$FieldIndexer.processValue(AutoTypeColumnIndexer.java:745)
~[druid-processing-33.0.0.jar:33.0.0]
at
org.apache.druid.segment.AutoTypeColumnIndexer$1.processField(AutoTypeColumnIndexer.java:105)
~[druid-processing-33.0.0.jar:33.0.0]
at
org.apache.druid.segment.nested.StructuredDataProcessor.processFields(StructuredDataProcessor.java:73)
~[druid-processing-33.0.0.jar:33.0.0]
at
org.apache.druid.segment.AutoTypeColumnIndexer.processAuto(AutoTypeColumnIndexer.java:213)
~[druid-processing-33.0.0.jar:33.0.0]
at
org.apache.druid.segment.AutoTypeColumnIndexer.processRowValsToUnsortedEncodedKeyComponent(AutoTypeColumnIndexer.java:160)
~[druid-processing-33.0.0.jar:33.0.0]
at
org.apache.druid.segment.incremental.IncrementalIndex.toIncrementalIndexRow(IncrementalIndex.java:560)
~[druid-processing-33.0.0.jar:33.0.0]
at
org.apache.druid.segment.incremental.IncrementalIndex.add(IncrementalIndex.java:493)
~[druid-processing-33.0.0.jar:33.0.0]
at
org.apache.druid.indexing.overlord.sampler.InputSourceSampler.sample(InputSourceSampler.java:174)
~[druid-indexing-service-33.0.0.jar:33.0.0]
... 81 more
```
When describing the datasource in question, there is only one complex data
type: `COMPLEX<quantilesDoublesSketch>`
It seems to be just the sampler that fails. By manually submitting a job
with an appropriately crafted spec (including an aggregation of the
quantilesDoublesSketch field), the job succeeds.
e.g. including this in the `metricsSpec` is fine:
```
{
"type": "quantilesDoublesSketch",
"name": "ds_ResponseTime",
"fieldName": "ds_ResponseTime",
"k": 1024,
"maxStreamLength": 1000000000
},
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
