zenfenan opened a new issue #10254:
URL: https://github.com/apache/druid/issues/10254
I updated my Kafka supervisor spec to set forceTimeChunkLock to false so
that I can run compaction tasks for intervals which has realtime segments but
the compaction task is stuck in `WAITING` state and overlord logs indicate it
is unable to acquire a lock.
### Affected Version
0.18.1
### Description
We have a Kafka supervisor with the query granularity at hourly level and
segment granularity at day level. And both of them are period granularity types
with IST as the timezone. Please refer the following granularitySpec.
```
"granularitySpec": {
"type": "uniform",
"queryGranularity": {
"type": "period",
"period": "PT1H",
"timeZone": "Asia/Kolkata"
},
"segmentGranularity": {
"type": "period",
"period": "P1D",
"timeZone": "Asia/Kolkata"
}
}
```
And since we have the segment granularity as `P1D`, we can only compact
segments at T-1 date since there will be realtime segments for today's
interval. I was reading through Druid's [task
reference](https://druid.apache.org/docs/latest/ingestion/tasks.html) page
which mentioned that if we enforce the locking type to be segment lock instead
of time lock, we can do compactions even for the interval which has realtime
segments. So I updated my Kafka indexing spec to set `forceTimeChunkLock` to
false under `context` field. However, even after updating to use segment lock,
when I submitted a compaction spec, it was in `WAITING` state. Reading through
the overlord.log, I found the following.
```
2020-08-07T15:34:00,529 INFO [qtp786047074-85]
org.apache.druid.indexing.overlord.MetadataTaskStorage - Inserting task
compact_kafka_ist_hourly_emingfdh_2020-08-07T15:34:00.527Z with status:
TaskStatus{id=compact_kafka_ist_hourly_emingfdh_2020-08-07T15:34:00.527Z,
status=RUNNING, duration=-1, errorMsg=null}
2020-08-07T15:34:00,531 INFO [qtp786047074-85]
org.apache.druid.indexing.overlord.TaskLockbox - Adding
task[compact_kafka_ist_hourly_emingfdh_2020-08-07T15:34:00.527Z] to activeTasks
2020-08-07T15:34:00,535 INFO [TaskQueue-Manager]
org.apache.druid.indexing.common.task.AbstractBatchIndexTask - Detected
segmentGranularity change. Using timeChunk lock
2020-08-07T15:34:00,535 INFO [TaskQueue-Manager]
org.apache.druid.indexing.overlord.TaskLockbox - Cannot create a new
taskLockPosse for request[TimeChunkLockRequest{lockType=EXCLUSIVE,
groupId='compact_kafka_ist_hourly_emingfdh_2020-08-07T15:34:00.527Z',
dataSource='kafka_ist_hourly',
interval=2020-08-07T00:00:00.000+05:30/2020-08-08T00:00:00.000+05:30,
preferredVersion='null', priority=25, revoked=false}] because existing
locks[[TaskLockPosse{taskLock=SegmentLock{lockType=EXCLUSIVE,
groupId='index_kafka_kafka_ist_hourly', dataSource='kafka_ist_hourly',
interval=2020-08-06T18:30:00.000Z/2020-08-07T18:30:00.000Z,
version='2020-08-06T18:29:13.134Z', partitionId=60, priority=75,
revoked=false},
taskIds=[index_kafka_kafka_ist_hourly_4c1beb3f5b90384_kmcljcgk]},
TaskLockPosse{taskLock=SegmentLock{lockType=EXCLUSIVE,
groupId='index_kafka_kafka_ist_hourly', dataSource='kafka_ist_hourly',
interval=2020-08-06T18:30:00.000Z/2020-08-07T18:30:00.000Z,
version='2020-08-06T18:29:13.134Z', partit
ionId=61, priority=75, revoked=false},
taskIds=[index_kafka_kafka_ist_hourly_505a81616efe052_ncbdikid]},
TaskLockPosse{taskLock=SegmentLock{lockType=EXCLUSIVE,
groupId='index_kafka_kafka_ist_hourly', dataSource='kafka_ist_hourly',
interval=2020-08-06T18:30:00.000Z/2020-08-07T18:30:00.000Z,
version='2020-08-06T18:29:13.134Z', partitionId=62, priority=75,
revoked=false},
taskIds=[index_kafka_kafka_ist_hourly_150349ea52a197b_gdnfljjg]},
TaskLockPosse{taskLock=SegmentLock{lockType=EXCLUSIVE,
groupId='index_kafka_kafka_ist_hourly', dataSource='kafka_ist_hourly',
interval=2020-08-06T18:30:00.000Z/2020-08-07T18:30:00.000Z,
version='2020-08-06T18:29:13.134Z', partitionId=63, priority=75,
revoked=false},
taskIds=[index_kafka_kafka_ist_hourly_7de877c9aee1a93_mefncmen]},
TaskLockPosse{taskLock=SegmentLock{lockType=EXCLUSIVE,
groupId='index_kafka_kafka_ist_hourly', dataSource='kafka_ist_hourly',
interval=2020-08-06T18:30:00.000Z/2020-08-07T18:30:00.000Z,
version='2020-08-06T18:29:13.134Z', part
itionId=64, priority=75, revoked=false},
taskIds=[index_kafka_kafka_ist_hourly_44ab741b2682141_anfcfpna]}]] have same or
higher priorities
```
So regardless of having forceTimeChunkLock set to false, it seems to enforce
time chunk lock and also that it fails to acquire a lock.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
