tanisdlj edited a comment on issue #10866: URL: https://github.com/apache/druid/issues/10866#issuecomment-814190073
Middlemanager log: ``` Apr 6 09:00:01 druid-dataserver-11 java[6601]: 2021-04-06T09:00:01,455 INFO [NodeRoleWatcher[COORDINATOR]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeRoleWatcher - Node[http://druid-master-1:8081] of role[coordinator] went offline. Apr 6 09:04:17 druid-dataserver-11 java[6601]: 2021-04-06T09:04:17,958 INFO [NodeRoleWatcher[COORDINATOR]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeRoleWatcher - Node[http://druid-master-1:8081] of role[coordinator] detected. Apr 6 09:23:34 druid-dataserver-11 java[6601]: 2021-04-06T09:23:34,935 INFO [WorkerTaskManager-NoticeHandler] org.apache.druid.indexing.worker.WorkerTaskManager - Task[index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc] started. Apr 6 09:23:34 druid-dataserver-11 java[6601]: 2021-04-06T09:23:34,951 INFO [forking-task-runner-0] org.apache.druid.indexing.overlord.ForkingTaskRunner - Running command: java -cp /etc/druid/_common:/etc/druid/middleManager:/opt/druid/lib/jackson-databind-2.10.5.1.jar:/opt/druid/lib/fastutil-8.2.3.jar:/opt/druid/lib/commons-logging-1.1.1.jar:/opt/druid/lib/commons-text-1.3.jar:/opt/druid/lib/curator-client-4.3.0.jar:/opt/druid/lib/netty-handler-4.1.48.Final.jar:/opt/druid/lib/javax.el-3.0.0.jar:/opt/druid/lib/maven-settings-builder-3.1.1.jar:/opt/druid/lib/joni-2.1.27.jar:/opt/druid/lib/joda-time-2.10.5.jar:/opt/druid/lib/classmate-1.1.0.jar:/opt/druid/lib/asm-commons-7.1.jar:/opt/druid/lib/javax.activation-api-1.2.0.jar:/opt/druid/lib/spymemcached-2.12.3.jar:/opt/druid/lib/jetty-http-9.4.34.v20201102.jar:/opt/druid/lib/commons-math3-3.6.1.jar:/opt/druid/lib/tesla-aether-0.0.5.jar:/opt/druid/lib/commons-io-2.6.jar:/opt/druid/lib/jackson-core-2.10.2.jar:/opt/druid/lib/audience-an notations-0.5.0.jar:/opt/druid/lib/guice-servlet-4.1.0.jar:/opt/druid/lib/commons-collections-3.2.2.jar:/opt/druid/lib/javax.el-api-3.0.0.jar:/opt/druid/lib/maven-artifact-3.6.0.jar:/opt/druid/lib/avatica-server-1.17.0.jar:/opt/druid/lib/httpcore-4.4.11.jar:/opt/druid/lib/aether-api-0.9.0.M2.jar:/opt/druid/lib/checker-qual-2.5.7.jar:/opt/druid/lib/protobuf-java-3.11.0.jar:/opt/druid/lib/RoaringBitmap-0.9.0.jar:/opt/druid/lib/maven-model-builder-3.1.1.jar:/opt/druid/lib/jetty-util-9.4.34.v20201102.jar:/opt/druid/lib/aether-spi-0.9.0.M2.jar:/opt/druid/lib/jaxb-runtime-2.3.1.jar:/opt/druid/lib/guice-multibindings-4.1.0.jar:/opt/druid/lib/asm-tree-7.1.jar:/opt/druid/lib/janino-3.0.11.jar:/opt/druid/lib/jmespath-java-1.11.199.jar:/opt/druid/lib/druid-console-0.20.1.jar:/opt/druid/lib/netty-handler-proxy-4.1.48.Final.jar:/opt/druid/lib/xz-1.8.jar:/opt/druid/lib/derbynet-10.14.2.0.jar:/opt/druid/lib/druid-core-0.20.1.jar:/opt/druid/lib/plexus-utils-3.0.24.jar:/opt/druid/lib/jackson-module- guice-2.10.2.jar:/opt/druid/lib/druid-indexing-service-0.20.1.jar:/opt/druid/lib/netty-transport-native- Apr 6 09:23:34 druid-dataserver-11 java[6601]: epoll-4.1.48.Final-linux-x86_64.jar:/opt/druid/lib/istack-commons-runtime-3.0.7.jar:/opt/druid/lib/aws-java-sdk-ec2-1.11.199.jar:/opt/druid/lib/async-http-client-2.5.3.jar:/opt/druid/lib/esri-geometry-api-2.2.0.jar:/opt/druid/lib/log4j-1.2-api-2.8.2.jar:/opt/druid/lib/compress-lzf-1.0.4.jar:/opt/druid/lib/curator-framework-4.3.0.jar:/opt/druid/lib/google-http-client-1.26.0.jar:/opt/druid/lib/config-magic-0.9.jar:/opt/druid/lib/netty-transport-4.1.48.Final.jar:/opt/druid/lib/log4j-core-2.8.2.jar:/opt/druid/lib/disruptor-3.3.6.jar:/opt/druid/lib/jsr305-2.0.1.jar:/opt/druid/lib/jetty-servlet-9.4.34.v20201102.jar:/opt/druid/lib/aws-java-sdk-kms-1.11.199.jar:/opt/druid/lib/druid-aws-common-0.20.1.jar:/opt/druid/lib/avatica-metrics-1.17.0.jar:/opt/druid/lib/log4j-jul-2.8.2.jar:/opt/druid/lib/commons-compiler-3.0.11.jar:/opt/druid/lib/netty-common-4.1.48.Final.jar:/opt/druid/lib/aether-util-0.9.0.M2.jar:/opt/druid/lib/netty-resolver-dns-4.1 .48.Final.jar:/opt/druid/lib/calcite-core-1.21.0.jar:/opt/druid/lib/netty-buffer-4.1.48.Final.jar:/opt/druid/lib/druid-processing-0.20.1.jar:/opt/druid/lib/aws-java-sdk-core-1.11.199.jar:/opt/druid/lib/druid-hll-0.20.1.jar:/opt/druid/lib/calcite-linq4j-1.21.0.jar:/opt/druid/lib/jdbi-2.63.1.jar:/opt/druid/lib/maven-model-3.1.1.jar:/opt/druid/lib/resilience4j-core-1.3.1.jar:/opt/druid/lib/async-http-client-netty-utils-2.5.3.jar:/opt/druid/lib/commons-lang3-3.8.1.jar:/opt/druid/lib/jsr311-api-1.1.1.jar:/opt/druid/lib/druid-sql-0.20.1.jar:/opt/druid/lib/avatica-core-1.17.0.jar:/opt/druid/lib/extendedset-0.20.1.jar:/opt/druid/lib/google-api-client-1.26.0.jar:/opt/druid/lib/jetty-io-9.4.34.v20201102.jar:/opt/druid/lib/log4j-api-2.8.2.jar:/opt/druid/lib/aopalliance-1.0.jar:/opt/druid/lib/jaxb-api-2.3.1.jar:/opt/druid/lib/icu4j-55.1.jar:/opt/druid/lib/validation-api-1.1.0.Final.jar:/opt/druid/lib/resilience4j-bulkhead-1.3.1.jar:/opt/druid/lib/caffeine-2.8.0.jar:/opt/druid/lib/datasketches-m emory-1.2.0-incubating.jar:/opt/druid/lib/j2objc-annotations-1.1.jar:/opt/druid/lib/jetty-client-9.4.34. Apr 6 09:23:34 druid-dataserver-11 java[6601]: v20201102.jar:/opt/druid/lib/jboss-logging-3.2.1.Final.jar:/opt/druid/lib/shims-0.9.0.jar:/opt/druid/lib/jetty-servlets-9.4.34.v20201102.jar:/opt/druid/lib/wagon-provider-api-2.4.jar:/opt/druid/lib/reactive-streams-1.0.2.jar:/opt/druid/lib/jetty-rewrite-9.4.34.v20201102.jar:/opt/druid/lib/json-smart-2.3.jar:/opt/druid/lib/aether-connector-okhttp-0.0.9.jar:/opt/druid/lib/aws-java-sdk-s3-1.11.199.jar:/opt/druid/lib/stax-ex-1.8.jar:/opt/druid/lib/derby-10.14.2.0.jar:/opt/druid/lib/guice-4.1.0.jar:/opt/druid/lib/sigar-1.6.5.132.jar:/opt/druid/lib/jetty-security-9.4.34.v20201102.jar:/opt/druid/lib/jvm-attach-api-1.5.jar:/opt/druid/lib/opencsv-4.6.jar:/opt/druid/lib/commons-compress-1.19.jar:/opt/druid/lib/datasketches-java-1.3.0-incubating.jar:/opt/druid/lib/jackson-jq-0.0.10.jar:/opt/druid/lib/jersey-servlet-1.19.3.jar:/opt/druid/lib/druid-gcp-common-0.20.1.jar:/opt/druid/lib/aether-connector-file-0.9.0.M2.jar:/opt/druid/lib/derbyclient- 10.14.2.0.jar:/opt/druid/lib/jackson-module-jaxb-annotations-2.10.2.jar:/opt/druid/lib/maven-settings-3.1.1.jar:/opt/druid/lib/druid-indexing-hadoop-0.20.1.jar:/opt/druid/lib/airline-0.7.jar:/opt/druid/lib/jna-4.5.1.jar:/opt/druid/lib/javax.activation-1.2.0.jar:/opt/druid/lib/ion-java-1.0.2.jar:/opt/druid/lib/jackson-jaxrs-base-2.10.2.jar:/opt/druid/lib/slf4j-api-1.7.25.jar:/opt/druid/lib/jakarta.activation-api-1.2.1.jar:/opt/druid/lib/httpclient-4.5.10.jar:/opt/druid/lib/druid-server-0.20.1.jar:/opt/druid/lib/netty-3.10.6.Final.jar:/opt/druid/lib/jackson-datatype-joda-2.10.2.jar:/opt/druid/lib/asm-analysis-7.1.jar:/opt/druid/lib/maven-aether-provider-3.1.1.jar:/opt/druid/lib/log4j-slf4j-impl-2.8.2.jar:/opt/druid/lib/vavr-match-0.10.2.jar:/opt/druid/lib/metrics-core-4.0.0.jar:/opt/druid/lib/rhino-1.7.11.jar:/opt/druid/lib/aether-impl-0.9.0.M2.jar:/opt/druid/lib/netty-codec-http-4.1.48.Final.jar:/opt/druid/lib/netty-codec-4.1.48.Final.jar:/opt/druid/lib/jackson-annotations-2.10.2.jar :/opt/druid/lib/commons-net-3.6.jar:/opt/druid/lib/jackson-datatype-guava-2.10.2.jar:/opt/druid/lib/txw2 Apr 6 09:23:34 druid-dataserver-11 java[6601]: -2.3.1.jar:/opt/druid/lib/jackson-jaxrs-smile-provider-2.10.2.jar:/opt/druid/lib/google-oauth-client-1.26.0.jar:/opt/druid/lib/zstd-jni-1.3.3-1.jar:/opt/druid/lib/jersey-guice-1.19.3.jar:/opt/druid/lib/jcl-over-slf4j-1.7.12.jar:/opt/druid/lib/netty-codec-socks-4.1.48.Final.jar:/opt/druid/lib/lz4-java-1.7.1.jar:/opt/druid/lib/asm-7.1.jar:/opt/druid/lib/commons-lang-2.6.jar:/opt/druid/lib/jline-0.9.94.jar:/opt/druid/lib/commons-collections4-4.2.jar:/opt/druid/lib/javax.inject-1.jar:/opt/druid/lib/netty-resolver-4.1.48.Final.jar:/opt/druid/lib/curator-x-discovery-4.3.0.jar:/opt/druid/lib/jersey-core-1.19.3.jar:/opt/druid/lib/maven-repository-metadata-3.1.1.jar:/opt/druid/lib/json-path-2.3.0.jar:/opt/druid/lib/guava-16.0.1.jar:/opt/druid/lib/netty-codec-dns-4.1.48.Final.jar:/opt/druid/lib/commons-beanutils-1.9.4.jar:/opt/druid/lib/jackson-dataformat-cbor-2.10.2.jar:/opt/druid/lib/plexus-interpolation-1.19.jar:/opt/druid/lib/okhttp-1.0.2. jar:/opt/druid/lib/hibernate-validator-5.2.5.Final.jar:/opt/druid/lib/google-http-client-jackson2-1.26.0.jar:/opt/druid/lib/javax.servlet-api-3.1.0.jar:/opt/druid/lib/commons-dbcp2-2.0.1.jar:/opt/druid/lib/jakarta.xml.bind-api-2.3.2.jar:/opt/druid/lib/commons-pool2-2.2.jar:/opt/druid/lib/jetty-continuation-9.4.34.v20201102.jar:/opt/druid/lib/jcodings-1.0.43.jar:/opt/druid/lib/jetty-proxy-9.4.34.v20201102.jar:/opt/druid/lib/commons-codec-1.13.jar:/opt/druid/lib/jetty-server-9.4.34.v20201102.jar:/opt/druid/lib/error_prone_annotations-2.3.2.jar:/opt/druid/lib/aggdesigner-algorithm-6.0.jar:/opt/druid/lib/netty-reactive-streams-2.0.0.jar:/opt/druid/lib/jackson-jaxrs-json-provider-2.10.2.jar:/opt/druid/lib/FastInfoset-1.2.15.jar:/opt/druid/lib/jackson-dataformat-smile-2.10.2.jar:/opt/druid/lib/netty-transport-native-unix-common-4.1.48.Final.jar:/opt/druid/lib/curator-recipes-4.3.0.jar:/opt/druid/lib/vavr-0.10.2.jar:/opt/druid/lib/antlr4-runtime-4.5.1.jar:/opt/druid/lib/jersey-server-1.19. 3.jar:/opt/druid/lib/guice-assistedinject-4.1.0.jar:/opt/druid/lib/accessors-smart-1.2.jar:/opt/druid/li Apr 6 09:23:34 druid-dataserver-11 java[6601]: b/druid-services-0.20.1.jar:/opt/druid/lib/zookeeper-3.4.14.jar:/opt/jmx-exporter/jmx_prometheus_javaagent-0.14.0.jar -server -Xmx3g -Xms3g -XX:MaxDirectMemorySize=2g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Djava.io.tmpdir=/druid/tmp/peon -XX:+UseG1GC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=50 -XX:GCLogFileSize=10m -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/druid/peon.hprof -Xloggc:/var/log/druid/peon.gc.log -Ddruid.announcer.skipLoadSpec=true -Ddruid.indexer.task.baseTaskDir=/druid/task -Ddruid.metadata.storage.connector.password=<masked> -Ddruid.indexer.fork.property.druid.processing.numThreads=2 -Ddruid.emitter.logging.logLevel=info -Ddruid.indexer.logs.prefix=indexing-logs -Ddruid.emitter=http -Ddruid.indexer.fork.property.druid.processing.buffer.sizeBytes=104857600 -Ddruid.google.bucket=seedtag_analytics_druid -Ddruid.indexer.task.restoreTasksOnRestart=true -Duser.timezone=UTC -Ddruid.cache.sizeInBytes=300MiB -Ddruid.announcer.skipDimensionsAndMetrics=true -Ddruid.emitter.http.recipientBaseUrl=http://druid-exporter-1:8080/druid -Dfile.encoding.pkg=sun.io -Ddruid.selectors.coordinator.serviceName=druid/coordinator -Ddruid.selectors.indexing.serviceName=druid/overlord -Ddruid.indexing.doubleStorage=double -Ddruid.query.vectorize=true -Ddruid.realtime.cache.useCache=true -Ddruid.server.http.numThreads=50 -Ddruid.worker.capacity=9 -Ddruid.cache.type=caffeine -Ddruid.google.prefix=storage -Ddruid.service=druid/middleManager -Ddruid.metadata.storage.connector.user=druid -Ddruid.metadata.storage.type=mysql -Ddruid.metadata.storage.connector.connectURI=jdbc:mysql://10.167.144.13:3306/druid -Ddruid.plaintextPort=8091 -Djava.io.tmpdir=/druid/tmp/middlemanager -Ddruid.extensions.loadList=["druid-datasketches", "druid-lookups-cached-global", "druid-google-extensi Apr 6 09:23:34 druid-dataserver-11 java[6601]: ons", "mysql-metadata-storage", "druid-kafka-indexing-service", "ssp-events-extension"] -Ddruid.startup.logging.logProperties=true -Ddruid.indexer.fork.property.druid.lookup.enableLookupSyncOnStartup=true -Ddruid.global.http.readTimeout=PT30M -Ddruid.zk.service.host=druid-zookeeper-1:2181,druid-zookeeper-2:2181,druid-zookeeper-3:2181 -Ddruid.realtime.cache.populateCache=true -Ddruid.global.http.unusedConnectionTimeout=PT27M -Ddruid.indexer.logs.bucket=seedtag_analytics_druid_logs -Ddruid.zk.paths.base=/druid -Dfile.encoding=UTF-8 -Ddruid.storage.type=google -Ddruid.indexer.fork.property.druid.processing.numMergeBuffers=2 -Ddruid.indexer.logs.type=google -Ddruid.processing.numThreads=2 -Ddruid.processing.buffer.sizeBytes=104857600 -Ddruid.lookup.enableLookupSyncOnStartup=true -Ddruid.processing.numMergeBuffers=2 -Ddruid.metrics.emitter.dimension.dataSource=ssp-events-hourly -Ddruid.metrics.emitter.dimension.taskId=index_kafka_ssp-even ts-hourly_4e62ca9663a43dc_cojlbdpc -Ddruid.metrics.emitter.dimension.taskType=index_kafka -Ddruid.host=druid-dataserver-11 -Ddruid.plaintextPort=8100 -Ddruid.tlsPort=-1 -Ddruid.task.executor.service=druid/middleManager -Ddruid.task.executor.host=druid-dataserver-11 -Ddruid.task.executor.plaintextPort=8091 -Ddruid.task.executor.enablePlaintextPort=true -Ddruid.task.executor.tlsPort=-1 -Ddruid.task.executor.enableTlsPort=false org.apache.druid.cli.Main internal peon /druid/task/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc/task.json /druid/task/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc/7c3b599e-f705-4066-b0b6-672a30c8dc30/status.json /druid/task/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc/7c3b599e-f705-4066-b0b6-672a30c8dc30/report.json --loadBroadcastSegments true Apr 6 09:23:34 druid-dataserver-11 java[6601]: 2021-04-06T09:23:34,992 INFO [forking-task-runner-0] org.apache.druid.indexing.overlord.ForkingTaskRunner - Logging task index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc output to: /druid/task/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc/log Apr 6 09:24:14 druid-dataserver-11 java[6601]: 2021-04-06T09:24:14,274 INFO [WorkerTaskManager-NoticeHandler] org.apache.druid.indexing.worker.WorkerTaskManager - Task[index_kafka_ssp-events-hourly_210a1b80ffd27f0_ilblpfpb] started. Apr 6 09:24:14 druid-dataserver-11 java[6601]: 2021-04-06T09:24:14,276 INFO [forking-task-runner-8] org.apache.druid.indexing.overlord.ForkingTaskRunner - Running command: java -cp /etc/druid/_common:/etc/druid/middleManager:/opt/druid/lib/jackson-databind-2.10.5.1.jar:/opt/druid/lib/fastutil-8.2.3.jar:/opt/druid/lib/commons-logging-1.1.1.jar:/opt/druid/lib/commons-text-1.3.jar:/opt/druid/lib/curator-client-4.3.0.jar:/opt/druid/lib/netty-handler-4.1.48.Final.jar:/opt/druid/lib/javax.el-3.0.0.jar:/opt/druid/lib/maven-settings-builder-3.1.1.jar:/opt/druid/lib/joni-2.1.27.jar:/opt/druid/lib/joda-time-2.10.5.jar:/opt/druid/lib/classmate-1.1.0.jar:/opt/druid/lib/asm-commons-7.1.jar:/opt/druid/lib/javax.activation-api-1.2.0.jar:/opt/druid/lib/spymemcached-2.12.3.jar:/opt/druid/lib/jetty-http-9.4.34.v20201102.jar:/opt/druid/lib/commons-math3-3.6.1.jar:/opt/druid/lib/tesla-aether-0.0.5.jar:/opt/druid/lib/commons-io-2.6.jar:/opt/druid/lib/jackson-core-2.10.2.jar:/opt/druid/lib/audience-an notations-0.5.0.jar:/opt/druid/lib/guice-servlet-4.1.0.jar:/opt/druid/lib/commons-collections-3.2.2.jar:/opt/druid/lib/javax.el-api-3.0.0.jar:/opt/druid/lib/maven-artifact-3.6.0.jar:/opt/druid/lib/avatica-server-1.17.0.jar:/opt/druid/lib/httpcore-4.4.11.jar:/opt/druid/lib/aether-api-0.9.0.M2.jar:/opt/druid/lib/checker-qual-2.5.7.jar:/opt/druid/lib/protobuf-java-3.11.0.jar:/opt/druid/lib/RoaringBitmap-0.9.0.jar:/opt/druid/lib/maven-model-builder-3.1.1.jar:/opt/druid/lib/jetty-util-9.4.34.v20201102.jar:/opt/druid/lib/aether-spi-0.9.0.M2.jar:/opt/druid/lib/jaxb-runtime-2.3.1.jar:/opt/druid/lib/guice-multibindings-4.1.0.jar:/opt/druid/lib/asm-tree-7.1.jar:/opt/druid/lib/janino-3.0.11.jar:/opt/druid/lib/jmespath-java-1.11.199.jar:/opt/druid/lib/druid-console-0.20.1.jar:/opt/druid/lib/netty-handler-proxy-4.1.48.Final.jar:/opt/druid/lib/xz-1.8.jar:/opt/druid/lib/derbynet-10.14.2.0.jar:/opt/druid/lib/druid-core-0.20.1.jar:/opt/druid/lib/plexus-utils-3.0.24.jar:/opt/druid/lib/jackson-module- guice-2.10.2.jar:/opt/druid/lib/druid-indexing-service-0.20.1.jar:/opt/druid/lib/netty-transport-native- Apr 6 09:24:14 druid-dataserver-11 java[6601]: epoll-4.1.48.Final-linux-x86_64.jar:/opt/druid/lib/istack-commons-runtime-3.0.7.jar:/opt/druid/lib/aws-java-sdk-ec2-1.11.199.jar:/opt/druid/lib/async-http-client-2.5.3.jar:/opt/druid/lib/esri-geometry-api-2.2.0.jar:/opt/druid/lib/log4j-1.2-api-2.8.2.jar:/opt/druid/lib/compress-lzf-1.0.4.jar:/opt/druid/lib/curator-framework-4.3.0.jar:/opt/druid/lib/google-http-client-1.26.0.jar:/opt/druid/lib/config-magic-0.9.jar:/opt/druid/lib/netty-transport-4.1.48.Final.jar:/opt/druid/lib/log4j-core-2.8.2.jar:/opt/druid/lib/disruptor-3.3.6.jar:/opt/druid/lib/jsr305-2.0.1.jar:/opt/druid/lib/jetty-servlet-9.4.34.v20201102.jar:/opt/druid/lib/aws-java-sdk-kms-1.11.199.jar:/opt/druid/lib/druid-aws-common-0.20.1.jar:/opt/druid/lib/avatica-metrics-1.17.0.jar:/opt/druid/lib/log4j-jul-2.8.2.jar:/opt/druid/lib/commons-compiler-3.0.11.jar:/opt/druid/lib/netty-common-4.1.48.Final.jar:/opt/druid/lib/aether-util-0.9.0.M2.jar:/opt/druid/lib/netty-resolver-dns-4.1 .48.Final.jar:/opt/druid/lib/calcite-core-1.21.0.jar:/opt/druid/lib/netty-buffer-4.1.48.Final.jar:/opt/druid/lib/druid-processing-0.20.1.jar:/opt/druid/lib/aws-java-sdk-core-1.11.199.jar:/opt/druid/lib/druid-hll-0.20.1.jar:/opt/druid/lib/calcite-linq4j-1.21.0.jar:/opt/druid/lib/jdbi-2.63.1.jar:/opt/druid/lib/maven-model-3.1.1.jar:/opt/druid/lib/resilience4j-core-1.3.1.jar:/opt/druid/lib/async-http-client-netty-utils-2.5.3.jar:/opt/druid/lib/commons-lang3-3.8.1.jar:/opt/druid/lib/jsr311-api-1.1.1.jar:/opt/druid/lib/druid-sql-0.20.1.jar:/opt/druid/lib/avatica-core-1.17.0.jar:/opt/druid/lib/extendedset-0.20.1.jar:/opt/druid/lib/google-api-client-1.26.0.jar:/opt/druid/lib/jetty-io-9.4.34.v20201102.jar:/opt/druid/lib/log4j-api-2.8.2.jar:/opt/druid/lib/aopalliance-1.0.jar:/opt/druid/lib/jaxb-api-2.3.1.jar:/opt/druid/lib/icu4j-55.1.jar:/opt/druid/lib/validation-api-1.1.0.Final.jar:/opt/druid/lib/resilience4j-bulkhead-1.3.1.jar:/opt/druid/lib/caffeine-2.8.0.jar:/opt/druid/lib/datasketches-m emory-1.2.0-incubating.jar:/opt/druid/lib/j2objc-annotations-1.1.jar:/opt/druid/lib/jetty-client-9.4.34. -- Apr 6 13:25:16 druid-dataserver-11 java[6601]: 2021-04-06T13:25:16,870 INFO [forking-task-runner-3] org.apache.druid.indexing.overlord.ForkingTaskRunner - Logging task index_kafka_ssp-events-hourly_94de22349e72f81_elfcdclf output to: /druid/task/index_kafka_ssp-events-hourly_94de22349e72f81_elfcdclf/log Apr 6 13:30:01 druid-dataserver-11 java[6601]: 2021-04-06T13:30:01,087 INFO [NodeRoleWatcher[COORDINATOR]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeRoleWatcher - Node[http://druid-master-2:8081] of role[coordinator] went offline. Apr 6 13:33:45 druid-dataserver-11 java[6601]: 2021-04-06T13:33:45,452 INFO [NodeRoleWatcher[COORDINATOR]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeRoleWatcher - Node[http://druid-master-2:8081] of role[coordinator] detected. Apr 6 13:37:09 druid-dataserver-11 java[6601]: 2021-04-06T13:37:09,618 INFO [forking-task-runner-0-[index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc]] org.apache.druid.indexing.overlord.ForkingTaskRunner - Process exited with status[0] for task: index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc Apr 6 13:37:09 druid-dataserver-11 java[6601]: 2021-04-06T13:37:09,618 INFO [forking-task-runner-0] org.apache.druid.storage.google.GoogleTaskLogs - Pushing task log /druid/task/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc/log to: indexing-logs/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc Apr 6 13:37:09 druid-dataserver-11 java[6601]: 2021-04-06T13:37:09,920 INFO [forking-task-runner-0] org.apache.druid.storage.google.GoogleTaskLogs - Pushing task reports /druid/task/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc/7c3b599e-f705-4066-b0b6-672a30c8dc30/report.json to: indexing-logs/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc.report.json Apr 6 13:37:10 druid-dataserver-11 java[6601]: 2021-04-06T13:37:10,069 INFO [forking-task-runner-0] org.apache.druid.indexing.overlord.ForkingTaskRunner - Removing task directory: /druid/task/index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc Apr 6 13:37:10 druid-dataserver-11 java[6601]: 2021-04-06T13:37:10,128 INFO [WorkerTaskManager-NoticeHandler] org.apache.druid.indexing.worker.WorkerTaskManager - Task [index_kafka_ssp-events-hourly_4e62ca9663a43dc_cojlbdpc] completed with status [FAILED]. Apr 6 13:38:00 druid-dataserver-11 java[6601]: 2021-04-06T13:38:00,818 INFO [forking-task-runner-8-[index_kafka_ssp-events-hourly_210a1b80ffd27f0_ilblpfpb]] org.apache.druid.indexing.overlord.ForkingTaskRunner - Process exited with status[0] for task: index_kafka_ssp-events-hourly_210a1b80ffd27f0_ilblpfpb Apr 6 13:38:00 druid-dataserver-11 java[6601]: 2021-04-06T13:38:00,819 INFO [forking-task-runner-8] org.apache.druid.storage.google.GoogleTaskLogs - Pushing task log /druid/task/index_kafka_ssp-events-hourly_210a1b80ffd27f0_ilblpfpb/log to: indexing-logs/index_kafka_ssp-events-hourly_210a1b80ffd27f0_ilblpfpb Apr 6 13:38:01 druid-dataserver-11 java[6601]: 2021-04-06T13:38:01,086 INFO [forking-task-runner-8] org.apache.druid.storage.google.GoogleTaskLogs - Pushing task reports /druid/task/index_kafka_ssp-events-hourly_210a1b80ffd27f0_ilblpfpb/64b72335-39b4-40b6-b9b9-daf6ab7f7ed8/report.json to: indexing-logs/index_kafka_ssp-events-hourly_210a1b80ffd27f0_ilblpfpb.report.json ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
