[
https://issues.apache.org/jira/browse/HIVE-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16116339#comment-16116339
]
Yuming Wang commented on HIVE-17172:
------------------------------------
Seems this patch cause {{llap_uncompressed}} test failed:
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed]
(batchId=56)
{noformat}
logs:
{noformat}
2017-08-07T02:02:51,187 INFO [LocalJobRunner Map Task Executor #0] LlapIoImpl:
Llap counters: Fragment counters for [wangyuming01/192.16
8.77.55, warehouse.orc_llap,
pfile:/root/dev/hive/itests/qtest/target/warehouse/orc_llap/000000_0
([-4890527034409466364, 1502096569000,
473356]), 0,1]: [ NUM_VECTOR_BATCHES=0, NUM_DECODED_BATCHES=0,
SELECTED_ROWGROUPS=13, NUM_ERRORS=1, ROWS_EMITTED=0, METADATA_CACHE_HIT=0,
METADATA_CACHE_MISS=2, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=466121,
ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=18839425,
DECODE_TIME_NS=0, HDFS_TIME_NS=11545686, CONSUMER_TIME_NS=0 ]
2017-08-07T02:02:51,187 DEBUG [LocalJobRunner Map Task Executor #0]
encoded.OrcEncodedDataReader: Encoded reader is being stopped
2017-08-07T02:02:51,187 INFO [LocalJobRunner Map Task Executor #0]
mapred.MapTask: Ignoring exception during close for org.apache.hadoop
.mapred.MapTask$TrackedRecordReader@4f92e7b2
java.io.IOException: java.lang.AssertionError: Elements not in order data range
[5882, 472003), size: 466121 type: array-backed and data
range [6070, 472003), size: 465933 type: array-backed; trying to insert into
[{range start: -1 end: -1}, {data range [5882, 472003), size
: 466121 type: array-backed}]
at
org.apache.hadoop.hive.llap.io.api.impl.LlapRecordReader.rethrowErrorIfAny(LlapRecordReader.java:349)
~[hive-llap-server-3.0.0
-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.llap.io.api.impl.LlapRecordReader.close(LlapRecordReader.java:340)
~[hive-llap-server-3.0.0-SNAPSHOT.ja
r:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doClose(CombineHiveRecordReader.java:135)
~[hive-exec-3.0.0-SNAPSHOT.jar:
3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.close(HiveContextAwareRecordReader.java:104)
~[hive-exec-3.0.0-SNAPS
HOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.close(HadoopShimsSecure.java:173)
~[hive-shims-common-3
.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.close(MapTask.java:210)
~[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:1981)
[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:468)
[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
[hadoop-mapreduce-client-core-2.8.0.jar:?]
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
[hadoop-mapreduce-client-common-2.8.0
.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_77]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_77]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_77]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_77]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
Caused by: java.lang.AssertionError: Elements not in order data range [5882,
472003), size: 466121 type: array-backed and data range [607
0, 472003), size: 465933 type: array-backed; trying to insert into [{range
start: -1 end: -1}, {data range [5882, 472003), size: 466121 t
ype: array-backed}]
at
org.apache.hadoop.hive.common.io.DiskRangeList.assertInvalidOrder(DiskRangeList.java:64)
~[hive-storage-api-3.0.0-SNAPSHOT.jar
:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.common.io.DiskRangeList.checkOrder(DiskRangeList.java:56)
~[hive-storage-api-3.0.0-SNAPSHOT.jar:3.0.0-S
NAPSHOT]
at
org.apache.hadoop.hive.common.io.DiskRangeList.insertAfter(DiskRangeList.java:126)
~[hive-storage-api-3.0.0-SNAPSHOT.jar:3.0.0
-SNAPSHOT]
at
org.apache.hadoop.hive.common.io.DiskRangeList.split(DiskRangeList.java:183)
~[hive-storage-api-3.0.0-SNAPSHOT.jar:3.0.0-SNAPS
HOT]
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.preReadUncompressedStream(EncodedReaderImpl.java:1016)
~[hive-exec-
3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:382)
~[hive-exec-3.0.0-SN
APSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:426)
~[hive-llap-server-
3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:250)
~[hive-llap-server-3.0.0-SNAP
SHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:247)
~[hive-llap-server-3.0.0-SNAP
SHOT.jar:3.0.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_77]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_77]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
~[hadoop-common-2.8.0.jar:?]
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:247)
~[hive-llap-server-3.0
.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:96)
~[hive-llap-server-3.0.
0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
~[tez-common-0.8.4.jar:0.8.4]
at
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
~[hiv
e-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
... 4 more
{noformat}
> add ordering checks to DiskRangeList
> ------------------------------------
>
> Key: HIVE-17172
> URL: https://issues.apache.org/jira/browse/HIVE-17172
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Fix For: 3.0.0, 2.4.0
>
> Attachments: HIVE-17172.01.patch, HIVE-17172.02.patch,
> HIVE-17172.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
