JoeHo0727 opened a new issue #11180:
URL: https://github.com/apache/druid/issues/11180
Druid 0.20.2
Spec:
```
{
"type": "kafka",
"spec": {
"dataSchema": {
"dataSource": "weee-tracking-up",
"timestampSpec": {
"column": "time",
"format": "posix",
"missingValue": null
},
"dimensionsSpec": {
"dimensions": [
{
"type": "long",
"name": "data_version",
"multiValueHandling": "SORTED_ARRAY",
"createBitmapIndex": false
},
{
"type": "string",
"name": "type",
"multiValueHandling": "SORTED_ARRAY",
"createBitmapIndex": true
}
],
"dimensionExclusions": [
"count",
"time"
]
},
"metricsSpec": [
{
"type": "count",
"name": "count"
}
],
"granularitySpec": {
"type": "uniform",
"segmentGranularity": "HOUR",
"queryGranularity": "MINUTE",
"rollup": true,
"intervals": null
},
"transformSpec": {
"filter": null,
"transforms": []
}
},
"ioConfig": {
"topic": "weee-tracking",
"inputFormat": {
"type": "json",
"flattenSpec": {
"useFieldDiscovery": true,
"fields": []
},
"featureSpec": {}
},
"replicas": 1,
"taskCount": 1,
"taskDuration": "PT3600S",
"consumerProperties": {
"bootstrap.servers":
"b-3.bigdata-kafka.lip1h3.c6.kafka.us-west-2.amazonaws.com:9092,b-2.bigdata-kafka.lip1h3.c6.kafka.us-west-2.amazonaws.com:9092,b-1.bigdata-kafka.lip1h3.c6.kafka.us-west-2.amazonaws.com:9092"
},
"pollTimeout": 100,
"startDelay": "PT5S",
"period": "PT30S",
"useEarliestOffset": false,
"completionTimeout": "PT1800S",
"lateMessageRejectionPeriod": null,
"earlyMessageRejectionPeriod": null,
"lateMessageRejectionStartDateTime": null,
"stream": "weee-tracking",
"useEarliestSequenceNumber": false,
"type": "kafka"
},
"tuningConfig": {
"type": "kafka",
"maxRowsInMemory": 1000000,
"maxBytesInMemory": 0,
"maxRowsPerSegment": 5000000,
"maxTotalRows": null,
"intermediatePersistPeriod": "PT10M",
"basePersistDirectory":
"/usr/local/apache-druid-0.20.2/var/tmp/druid-realtime-persist6790745498049571661",
"maxPendingPersists": 0,
"indexSpec": {
"bitmap": {
"type": "roaring",
"compressRunOnSerialization": true
},
"dimensionCompression": "lz4",
"metricCompression": "lz4",
"longEncoding": "longs",
"segmentLoader": null
},
"indexSpecForIntermediatePersists": {
"bitmap": {
"type": "roaring",
"compressRunOnSerialization": true
},
"dimensionCompression": "lz4",
"metricCompression": "lz4",
"longEncoding": "longs",
"segmentLoader": null
},
"buildV9Directly": true,
"reportParseExceptions": false,
"handoffConditionTimeout": 0,
"resetOffsetAutomatically": false,
"segmentWriteOutMediumFactory": null,
"workerThreads": null,
"chatThreads": null,
"chatRetries": 8,
"httpTimeout": "PT10S",
"shutdownTimeout": "PT80S",
"offsetFetchPeriod": "PT30S",
"intermediateHandoffPeriod": "P2147483647D",
"logParseExceptions": false,
"maxParseExceptions": 2147483647,
"maxSavedParseExceptions": 0,
"skipSequenceNumberAvailabilityCheck": false,
"repartitionTransitionDuration": "PT120S"
}
}
}
```
Error
```
2021-04-28T09:36:34,335 INFO
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.loading.LocalDataSegmentKiller - Deleting
segment[weee-tracking-up_2021-04-28T02:00:00.000Z_2021-04-28T03:00:00.000Z_2021-04-28T00:32:14.912Z_9]
from
directory[/usr/local/apache-druid-0.20.2/var/druid/segments/weee-tracking-up/2021-04-28T02:00:00.000Z_2021-04-28T03:00:00.000Z/2021-04-28T00:32:14.912Z/9/e55718bb-939f-4c62-8a5f-d2ec1841b2dc/index.zip].
2021-04-28T09:36:34,335 INFO
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.loading.LocalDataSegmentKiller - Deleting
segment[weee-tracking-up_2021-04-26T12:00:00.000Z_2021-04-26T13:00:00.000Z_2021-04-27T12:15:16.898Z_1]
from
directory[/usr/local/apache-druid-0.20.2/var/druid/segments/weee-tracking-up/2021-04-26T12:00:00.000Z_2021-04-26T13:00:00.000Z/2021-04-27T12:15:16.898Z/1/c1e08b6c-d827-4e6f-97c3-41c8f7a2e546/index.zip].
2021-04-28T09:36:34,335 INFO
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.loading.LocalDataSegmentKiller - Deleting
segment[weee-tracking-up_2021-04-28T06:00:00.000Z_2021-04-28T07:00:00.000Z_2021-04-28T03:42:53.545Z_6]
from
directory[/usr/local/apache-druid-0.20.2/var/druid/segments/weee-tracking-up/2021-04-28T06:00:00.000Z_2021-04-28T07:00:00.000Z/2021-04-28T03:42:53.545Z/6/5306b8b5-aab2-450c-91f6-7e5231ccaee9/index.zip].
2021-04-28T09:36:34,335 INFO
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.loading.LocalDataSegmentKiller - Deleting
segment[weee-tracking-up_2021-04-28T09:00:00.000Z_2021-04-28T10:00:00.000Z_2021-04-28T08:59:56.366Z]
from
directory[/usr/local/apache-druid-0.20.2/var/druid/segments/weee-tracking-up/2021-04-28T09:00:00.000Z_2021-04-28T10:00:00.000Z/2021-04-28T08:59:56.366Z/0/70d5160f-59f0-4276-879a-b4bfd40a267a/index.zip].
2021-04-28T09:36:34,336 INFO
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.loading.LocalDataSegmentKiller - Deleting
segment[weee-tracking-up_2021-04-28T04:00:00.000Z_2021-04-28T05:00:00.000Z_2021-04-28T03:42:24.439Z_6]
from
directory[/usr/local/apache-druid-0.20.2/var/druid/segments/weee-tracking-up/2021-04-28T04:00:00.000Z_2021-04-28T05:00:00.000Z/2021-04-28T03:42:24.439Z/6/d0144de5-56c4-467d-9a1f-a92e1687df29/index.zip].
2021-04-28T09:36:34,336 INFO
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.loading.LocalDataSegmentKiller - Deleting
segment[weee-tracking-up_2021-04-28T03:00:00.000Z_2021-04-28T04:00:00.000Z_2021-04-28T02:37:04.210Z_7]
from
directory[/usr/local/apache-druid-0.20.2/var/druid/segments/weee-tracking-up/2021-04-28T03:00:00.000Z_2021-04-28T04:00:00.000Z/2021-04-28T02:37:04.210Z/7/124e058c-8348-48bc-bfdd-f5d246e56669/index.zip].
2021-04-28T09:36:34,337 ERROR
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver - Failed
to publish segments:
[weee-tracking-up_2021-04-26T09:00:00.000Z_2021-04-26T10:00:00.000Z_2021-04-27T10:08:46.369Z_3,
weee-tracking-up_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-27T10:00:01.080Z_14,
weee-tracking-up_2021-04-28T01:00:00.000Z_2021-04-28T02:00:00.000Z_2021-04-28T00:13:13.170Z_9,
weee-tracking-up_2021-04-28T07:00:00.000Z_2021-04-28T08:00:00.000Z_2021-04-28T07:00:00.235Z_2,
weee-tracking-up_2021-04-27T19:00:00.000Z_2021-04-27T20:00:00.000Z_2021-04-27T18:37:00.731Z_15,
weee-tracking-up_2021-04-27T23:00:00.000Z_2021-04-28T00:00:00.000Z_2021-04-27T22:52:16.461Z_11,
weee-tracking-up_2021-04-28T08:00:00.000Z_2021-04-28T09:00:00.000Z_2021-04-28T08:00:00.235Z_1,
weee-tracking-up_2021-04-28T05:00:00.000Z_2021-04-28T06:00:00.000Z_2021-04-28T04:03:01.142Z_6,
weee-tracking-up_2021-04-27T09:00:0
0.000Z_2021-04-27T10:00:00.000Z_2021-04-27T08:59:46.032Z_9,
weee-tracking-up_2021-04-27T22:00:00.000Z_2021-04-27T23:00:00.000Z_2021-04-27T17:05:37.618Z_13,
weee-tracking-up_2021-04-28T02:00:00.000Z_2021-04-28T03:00:00.000Z_2021-04-28T00:32:14.912Z_9,
weee-tracking-up_2021-04-26T12:00:00.000Z_2021-04-26T13:00:00.000Z_2021-04-27T12:15:16.898Z_1,
weee-tracking-up_2021-04-28T06:00:00.000Z_2021-04-28T07:00:00.000Z_2021-04-28T03:42:53.545Z_6,
weee-tracking-up_2021-04-28T09:00:00.000Z_2021-04-28T10:00:00.000Z_2021-04-28T08:59:56.366Z,
weee-tracking-up_2021-04-28T04:00:00.000Z_2021-04-28T05:00:00.000Z_2021-04-28T03:42:24.439Z_6,
weee-tracking-up_2021-04-28T03:00:00.000Z_2021-04-28T04:00:00.000Z_2021-04-28T02:37:04.210Z_7]
2021-04-28T09:36:34,337 WARN
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver - Failed
publish (org.apache.druid.java.util.common.ISE: Failed to publish segments
because of [java.lang.RuntimeException: Aborting transaction!])
2021-04-28T09:36:34,337 WARN
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver - Failed
publish, not removing segments:
[weee-tracking-up_2021-04-26T09:00:00.000Z_2021-04-26T10:00:00.000Z_2021-04-27T10:08:46.369Z_3,
weee-tracking-up_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-27T10:00:01.080Z_14,
weee-tracking-up_2021-04-28T01:00:00.000Z_2021-04-28T02:00:00.000Z_2021-04-28T00:13:13.170Z_9,
weee-tracking-up_2021-04-28T07:00:00.000Z_2021-04-28T08:00:00.000Z_2021-04-28T07:00:00.235Z_2,
weee-tracking-up_2021-04-27T19:00:00.000Z_2021-04-27T20:00:00.000Z_2021-04-27T18:37:00.731Z_15,
weee-tracking-up_2021-04-27T23:00:00.000Z_2021-04-28T00:00:00.000Z_2021-04-27T22:52:16.461Z_11,
weee-tracking-up_2021-04-28T08:00:00.000Z_2021-04-28T09:00:00.000Z_2021-04-28T08:00:00.235Z_1,
weee-tracking-up_2021-04-28T05:00:00.000Z_2021-04-28T06:00:00.000Z_2021-04-28T04:03:01.142Z_6,
weee-tracking-up_2021-04-
27T09:00:00.000Z_2021-04-27T10:00:00.000Z_2021-04-27T08:59:46.032Z_9,
weee-tracking-up_2021-04-27T22:00:00.000Z_2021-04-27T23:00:00.000Z_2021-04-27T17:05:37.618Z_13,
weee-tracking-up_2021-04-28T02:00:00.000Z_2021-04-28T03:00:00.000Z_2021-04-28T00:32:14.912Z_9,
weee-tracking-up_2021-04-26T12:00:00.000Z_2021-04-26T13:00:00.000Z_2021-04-27T12:15:16.898Z_1,
weee-tracking-up_2021-04-28T06:00:00.000Z_2021-04-28T07:00:00.000Z_2021-04-28T03:42:53.545Z_6,
weee-tracking-up_2021-04-28T09:00:00.000Z_2021-04-28T10:00:00.000Z_2021-04-28T08:59:56.366Z,
weee-tracking-up_2021-04-28T04:00:00.000Z_2021-04-28T05:00:00.000Z_2021-04-28T03:42:24.439Z_6,
weee-tracking-up_2021-04-28T03:00:00.000Z_2021-04-28T04:00:00.000Z_2021-04-28T02:37:04.210Z_7]
2021-04-28T09:36:34,345 ERROR
[[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc]-publish]
org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Error
while publishing segments for sequenceNumber[SequenceMetadata{sequenceId=0,
sequenceName='index_kafka_weee-tracking-up_e655490ba7c42ad_0', assignments=[],
startOffsets={0=195210860}, exclusiveStartPartitions=[],
endOffsets={0=195504210}, sentinel=false, checkpointed=true}]
org.apache.druid.java.util.common.ISE: Failed to publish segments because of
[java.lang.RuntimeException: Aborting transaction!]
at
org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver.lambda$publishInBackground$8(BaseAppenderatorDriver.java:651)
~[druid-server-0.20.2.jar:0.20.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
2021-04-28T09:36:34,357 INFO [task-runner-0-priority-0]
org.apache.druid.server.coordination.BatchDataSegmentAnnouncer - Unannouncing
segment[weee-tracking-up_2021-04-26T09:00:00.000Z_2021-04-26T10:00:00.000Z_2021-04-27T10:08:46.369Z_3]
at
path[/druid/segments/localhost:8101/localhost:8101_indexer-executor__default_tier_2021-04-28T08:36:34.612Z_0a28fcb9ea48481fb3005d1dbeda5cee0]
```
```
2021-04-28T09:36:34,447 INFO [task-runner-0-priority-0]
org.apache.druid.server.coordination.BatchDataSegmentAnnouncer - Unannouncing
segment[weee-tracking-up_2021-04-28T03:00:00.000Z_2021-04-28T04:00:00.000Z_2021-04-28T02:37:04.210Z_7]
at
path[/druid/segments/localhost:8101/localhost:8101_indexer-executor__default_tier_2021-04-28T08:36:34.612Z_0a28fcb9ea48481fb3005d1dbeda5cee0]
2021-04-28T09:36:34,447 INFO [task-runner-0-priority-0]
org.apache.druid.curator.announcement.Announcer - Unannouncing
[/druid/segments/localhost:8101/localhost:8101_indexer-executor__default_tier_2021-04-28T08:36:34.612Z_0a28fcb9ea48481fb3005d1dbeda5cee0]
2021-04-28T09:36:34,474 INFO [task-runner-0-priority-0]
org.apache.druid.curator.announcement.Announcer - Unannouncing
[/druid/internal-discovery/PEON/localhost:8101]
2021-04-28T09:36:34,480 INFO [task-runner-0-priority-0]
org.apache.druid.curator.discovery.CuratorDruidNodeAnnouncer - Unannounced self
[{"druidNode":{"service":"druid/middleManager","host":"localhost","bindOnHost":false,"plaintextPort":8101,"port":-1,"tlsPort":-1,"enablePlaintextPort":true,"enableTlsPort":false},"nodeType":"peon","services":{"dataNodeService":{"type":"dataNodeService","tier":"_default_tier","maxSize":0,"type":"indexer-executor","priority":0},"lookupNodeService":{"type":"lookupNodeService","lookupTier":"__default"}}}].
2021-04-28T09:36:34,481 INFO [task-runner-0-priority-0]
org.apache.druid.curator.announcement.Announcer - Unannouncing
[/druid/announcements/localhost:8101]
2021-04-28T09:36:34,485 ERROR [task-runner-0-priority-0]
org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner -
Encountered exception while running task.
java.util.concurrent.ExecutionException:
org.apache.druid.java.util.common.ISE: Failed to publish segments because of
[java.lang.RuntimeException: Aborting transaction!]
at
com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
~[guava-16.0.1.jar:?]
at
com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
~[guava-16.0.1.jar:?]
at
com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
~[guava-16.0.1.jar:?]
at
org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:802)
~[druid-indexing-service-0.20.2.jar:0.20.2]
at
org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:267)
[druid-indexing-service-0.20.2.jar:0.20.2]
at
org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.run(SeekableStreamIndexTask.java:145)
[druid-indexing-service-0.20.2.jar:0.20.2]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:451)
[druid-indexing-service-0.20.2.jar:0.20.2]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:423)
[druid-indexing-service-0.20.2.jar:0.20.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.apache.druid.java.util.common.ISE: Failed to publish segments
because of [java.lang.RuntimeException: Aborting transaction!]
at
org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver.lambda$publishInBackground$8(BaseAppenderatorDriver.java:651)
~[druid-server-0.20.2.jar:0.20.2]
... 4 more
2021-04-28T09:36:34,493 INFO [task-runner-0-priority-0]
org.apache.druid.indexing.worker.executor.ExecutorLifecycle - Task completed
with status: {
"id" : "index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc",
"status" : "FAILED",
"duration" : 3601768,
"errorMsg" : "java.util.concurrent.ExecutionException:
org.apache.druid.java.util.common.ISE: Failed to publish se...",
"location" : {
"host" : null,
"port" : -1,
"tlsPort" : -1
}
}
2021-04-28T09:36:34,499 INFO [main]
org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle
[module] stage [ANNOUNCEMENTS]
2021-04-28T09:36:34,500 INFO [main]
org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle
[module] stage [SERVER]
2021-04-28T09:36:34,509 INFO [main]
org.eclipse.jetty.server.AbstractConnector - Stopped
ServerConnector@3dbcd2e0{HTTP/1.1, (http/1.1)}{0.0.0.0:8101}
2021-04-28T09:36:34,509 INFO [main] org.eclipse.jetty.server.session - node0
Stopped scavenging
2021-04-28T09:36:34,511 INFO [main]
org.eclipse.jetty.server.handler.ContextHandler - Stopped
o.e.j.s.ServletContextHandler@70f8d217{/,null,STOPPED}
2021-04-28T09:36:34,513 INFO [main]
org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle
[module] stage [NORMAL]
2021-04-28T09:36:34,513 INFO [main]
org.apache.druid.server.coordination.ZkCoordinator - Stopping ZkCoordinator for
[DruidServerMetadata{name='localhost:8101', hostAndPort='localhost:8101',
hostAndTlsPort='null', maxSize=0, tier='_default_tier', type=indexer-executor,
priority=0}]
2021-04-28T09:36:34,513 INFO [main]
org.apache.druid.server.coordination.SegmentLoadDropHandler - Stopping...
2021-04-28T09:36:34,513 INFO [main]
org.apache.druid.server.coordination.SegmentLoadDropHandler - Stopped.
2021-04-28T09:36:34,514 INFO [main]
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner - Starting
graceful shutdown of
task[index_kafka_weee-tracking-up_e655490ba7c42ad_mcjebcgc].
2021-04-28T09:36:34,514 INFO [main]
org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner -
Stopping forcefully (status: [PUBLISHING])
2021-04-28T09:36:34,538 INFO
[LookupExtractorFactoryContainerProvider-MainThread]
org.apache.druid.query.lookup.LookupReferencesManager - Lookup Management loop
exited. Lookup notices are not handled anymore.
2021-04-28T09:36:34,571 INFO [Curator-Framework-0]
org.apache.curator.framework.imps.CuratorFrameworkImpl -
backgroundOperationsLoop exiting
2021-04-28T09:36:34,575 INFO [main] org.apache.zookeeper.ZooKeeper -
Session: 0x1001fc0eefd0144 closed
2021-04-28T09:36:34,575 INFO [main-EventThread]
org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
0x1001fc0eefd0144
2021-04-28T09:36:34,576 INFO [main]
org.apache.druid.java.util.common.lifecycle.Lifecycle - Stopping lifecycle
[module] stage [INIT]
Finished peon task
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
