Seung-Hyun Cheong created HIVE-21194:
----------------------------------------

             Summary: DruidStorageHandler should set a version of segment to UTC
                 Key: HIVE-21194
                 URL: https://issues.apache.org/jira/browse/HIVE-21194
             Project: Hive
          Issue Type: Bug
          Components: Druid integration
            Reporter: Seung-Hyun Cheong
            Assignee: Seung-Hyun Cheong


h1. Exception while running a KillTask
{code:java}
2019-01-30T16:58:35,354 ERROR [task-runner-0-priority-0] 
io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running 
task[KillTask{id=kill_upload_2018-12-31T00:00:00.000Z_2019-02-05T00:00:00.000Z_2019-02-01T16:52:31.851Z,
 type=kill, dataSource=upload}]
io.druid.java.util.common.ISE: WTF?! Unused 
segment[upload_2019-01-01T00:00:00.000Z_2019-01-02T00:00:00.000Z_2019-01-31T01:12:32.289+09:00]
 has version[2019-01-31T01:12:32.289+09:00] > task 
version[2019-01-30T16:58:29.992Z]
        at io.druid.indexing.common.task.KillTask.run(KillTask.java:94) 
~[druid-indexing-service-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
        at 
io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:444)
 [druid-indexing-service-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
        at 
io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:416)
 [druid-indexing-service-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[?:1.8.0_112]
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[?:1.8.0_112]
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[?:1.8.0_112]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
{code}
 
h1. AS-IS
h3. [KillTask compares 
versions|https://github.com/apache/incubator-druid/blob/master/indexing-service/src/main/java/org/apache/druid/indexing/common/task/KillTask.java#L88]

 
{code:java}
if (unusedSegment.getVersion().compareTo(myLock.getVersion()) > 0) {
  throw new ISE(
  "WTF?! Unused segment[%s] has version[%s] > task version[%s]",
  unusedSegment.getId(),
  unusedSegment.getVersion(),
  myLock.getVersion()
  );
}
{code}
 

 
h3. [KillTask version (UTC, 
"{{2019-01-30T16:58:29.992Z}}{{")}}|https://github.com/apache/incubator-druid/blob/8eae26fd4e7572060d112864dd3d5f6a865b9c89/indexing-service/src/main/java/org/apache/druid/indexing/overlord/TaskLockbox.java#L593]

 
{code:java}
version = DateTimes.nowUtc().toString();
{code}
 

 
h3. [Segment version (UTC+9, 
"2019-01-31T01:12:32.289+09:00")|https://github.com/apache/hive/blob/master/druid-handler/src/java/org/apache/hadoop/hive/druid/DruidStorageHandler.java#L755]
{code:java}
jobProperties.put(DruidConstants.DRUID_SEGMENT_VERSION, new 
DateTime().toString());
{code}
 

 
h1. TO-BE

 

Because druid uses UTC only for now, DruidStorageHandler should set a version 
of segment to UTC.

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to