[
https://issues.apache.org/jira/browse/HIVE-12631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202798#comment-16202798
]
Sergey Shelukhin edited comment on HIVE-12631 at 10/12/17 11:27 PM:
--------------------------------------------------------------------
[~teddy.choi] left one small comment on RB, can be fixed on commit. Although I
guess some test fix will be needed for the above failures also.
Looks like a NPE
{noformat}
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
attempt_1507843602261_0001_233_01_000000_1:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row (tag=0)
{"key":{"reducesinkkey0":null},"value":{"_col0":-1069736047}}
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:283)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:237)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0)
{"key":{"reducesinkkey0":null},"value":{"_col0":-1069736047}}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:289)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:254)
... 15 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row (tag=0)
{"key":{"reducesinkkey0":null},"value":{"_col0":-1069736047}}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:357)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:279)
... 17 more
Caused by: java.lang.NullPointerException
{noformat}
Unfortunately I cannot see full callstack in logs
was (Author: sershe):
[~teddy.choi] left one small comment on RB, can be fixed on commit. Although I
guess some test fix will be needed for the above failures also.
Looks like a NPE
{noformat}
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
attempt_1507843602261_0001_233_01_000000_1:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row (tag=0)
{"key":{"reducesinkkey0":null},"value":{"_col0":-1069736047}}
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:283)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:237)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0)
{"key":{"reducesinkkey0":null},"value":{"_col0":-1069736047}}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:289)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:254)
... 15 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row (tag=0)
{"key":{"reducesinkkey0":null},"value":{"_col0":-1069736047}}
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:357)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:279)
... 17 more
Caused by: java.lang.NullPointerException
Unfortunately I cannot see full callstack in logs
> LLAP: support ORC ACID tables
> -----------------------------
>
> Key: HIVE-12631
> URL: https://issues.apache.org/jira/browse/HIVE-12631
> Project: Hive
> Issue Type: Bug
> Components: llap, Transactions
> Reporter: Sergey Shelukhin
> Assignee: Teddy Choi
> Attachments: HIVE-12631.1.patch, HIVE-12631.10.patch,
> HIVE-12631.10.patch, HIVE-12631.11.patch, HIVE-12631.11.patch,
> HIVE-12631.12.patch, HIVE-12631.13.patch, HIVE-12631.15.patch,
> HIVE-12631.16.patch, HIVE-12631.17.patch, HIVE-12631.18.patch,
> HIVE-12631.19.patch, HIVE-12631.2.patch, HIVE-12631.20.patch,
> HIVE-12631.21.patch, HIVE-12631.22.patch, HIVE-12631.23.patch,
> HIVE-12631.24.patch, HIVE-12631.25.patch, HIVE-12631.26.patch,
> HIVE-12631.27.patch, HIVE-12631.28.patch, HIVE-12631.29.patch,
> HIVE-12631.3.patch, HIVE-12631.4.patch, HIVE-12631.5.patch,
> HIVE-12631.6.patch, HIVE-12631.7.patch, HIVE-12631.8.patch,
> HIVE-12631.8.patch, HIVE-12631.9.patch
>
>
> LLAP uses a completely separate read path in ORC to allow for caching and
> parallelization of reads and processing. This path does not support ACID. As
> far as I remember ACID logic is embedded inside ORC format; we need to
> refactor it to be on top of some interface, if practical; or just port it to
> LLAP read path.
> Another consideration is how the logic will work with cache. The cache is
> currently low-level (CB-level in ORC), so we could just use it to read bases
> and deltas (deltas should be cached with higher priority) and merge as usual.
> We could also cache merged representation in future.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
