[
https://issues.apache.org/jira/browse/HIVE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15186026#comment-15186026
]
Sergey Shelukhin commented on HIVE-12558:
-----------------------------------------
The logs are still there. Here's one interesting exception:
{noformat}
Caused by: java.io.IOException: java.lang.NullPointerException: DAG index is
not set
at
org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat.getRecordReader(LlapInputFormat.java:117)
~[hive-llap-server-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:67)
~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at sun.reflect.GeneratedConstructorAccessor186.newInstance(Unknown
Source) ~[?:?]
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[?:1.7.0_45]
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
~[?:1.7.0_45]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:255)
~[hive-shims-common-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:216)
~[hive-shims-common-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:343)
~[hive-shims-common-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:680)
~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
~[hadoop-mapreduce-client-core-2.6.0.jar:?]
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
~[hadoop-mapreduce-client-core-2.6.0.jar:?]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
~[hadoop-mapreduce-client-core-2.6.0.jar:?]
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
~[hadoop-mapreduce-client-common-2.6.0.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[?:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[?:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[?:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[?:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) ~[?:1.7.0_45]
Caused by: java.lang.NullPointerException: DAG index is not set
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:208)
~[guava-14.0.1.jar:?]
at
org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.<init>(LlapInputFormat.java:160)
~[hive-llap-server-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat.getRecordReader(LlapInputFormat.java:115)
~[hive-llap-server-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:67)
~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at sun.reflect.GeneratedConstructorAccessor186.newInstance(Unknown
Source) ~[?:?]
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[?:1.7.0_45]
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
~[?:1.7.0_45]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:255)
~[hive-shims-common-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:216)
~[hive-shims-common-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:343)
~[hive-shims-common-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:680)
~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
~[hadoop-mapreduce-client-core-2.6.0.jar:?]
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
~[hadoop-mapreduce-client-core-2.6.0.jar:?]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
~[hadoop-mapreduce-client-core-2.6.0.jar:?]
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
~[hadoop-mapreduce-client-common-2.6.0.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[?:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[?:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[?:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[?:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) ~[?:1.7.0_45]
{noformat}
> LLAP: output QueryFragmentCounters somewhere
> --------------------------------------------
>
> Key: HIVE-12558
> URL: https://issues.apache.org/jira/browse/HIVE-12558
> Project: Hive
> Issue Type: Bug
> Components: llap
> Reporter: Sergey Shelukhin
> Assignee: Prasanth Jayachandran
> Attachments: HIVE-12558.1.patch, HIVE-12558.2.patch,
> HIVE-12558.3.patch, HIVE-12558.3.patch, HIVE-12558.wip.patch,
> sample-output.png, sample-output2.png
>
>
> Right now, LLAP logs counters for every fragment; most of them are IO related
> and could be very useful, they also include table names so that things like
> cache hit ratio, etc., could be calculated for every table.
> We need to output them to some metrics system (preserving the breakdown by
> table, possibly also adding query ID or even stage) so that they'd be usable
> without grep/sed/awk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
