[
https://issues.apache.org/jira/browse/HIVE-20719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646834#comment-16646834
]
Eugene Koifman commented on HIVE-20719:
---------------------------------------
Disabling vectorization doesn't actually fix it. The difference is that
Vectorized acid reader has an assert to make sure the file name and ROW__IDs in
the file match, non-vectorized doesn't (just produces bad data by not reading
all relevant delete events).
Here is what the state looks like on disk after the update statement.
Everything from the update stmt is written to bucket_0. This creates data skew
for insert events and will ignore delete events that apply other buckets at
read time.
{noformat}
acid_uap/ds=tomorrow/delta_0000001_0000001_0000/bucket_00001 [length: 702]
{"operation":0,"originalTransaction":1,"bucket":536936448,"rowId":0,"currentTransaction":1,"row":{"a":1,"b":"bah"}}
________________________________________________________________________________________________________________________
acid_uap/ds=tomorrow/delta_0000001_0000001_0000/bucket_00000 [length: 703]
{"operation":0,"originalTransaction":1,"bucket":536870912,"rowId":0,"currentTransaction":1,"row":{"a":2,"b":"yah"}}
________________________________________________________________________________________________________________________
acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00001 [length: 711]
{"operation":0,"originalTransaction":2,"bucket":536936448,"rowId":0,"currentTransaction":2,"row":{"a":1,"b":"bah"}}
________________________________________________________________________________________________________________________
acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000 [length: 695]
{"operation":0,"originalTransaction":2,"bucket":536870912,"rowId":0,"currentTransaction":2,"row":{"a":2,"b":"yah"}}
________________________________________________________________________________________________________________________
acid_uap/ds=tomorrow/delta_0000003_0000003_0000/bucket_00000 [length: 733]
{"operation":0,"originalTransaction":3,"bucket":536870912,"rowId":0,"currentTransaction":3,"row":{"a":2,"b":"fred"}}
{"operation":0,"originalTransaction":3,"bucket":536870912,"rowId":1,"currentTransaction":3,"row":{"a":1,"b":"fred"}}
________________________________________________________________________________________________________________________
acid_uap/ds=today/delta_0000003_0000003_0000/bucket_00000 [length: 733]
{"operation":0,"originalTransaction":3,"bucket":536870912,"rowId":0,"currentTransaction":3,"row":{"a":2,"b":"fred"}}
{"operation":0,"originalTransaction":3,"bucket":536870912,"rowId":1,"currentTransaction":3,"row":{"a":1,"b":"fred"}}
________________________________________________________________________________________________________________________
acid_uap/ds=tomorrow/delete_delta_0000003_0000003_0000/bucket_00000 [length:
713]
{"operation":2,"originalTransaction":1,"bucket":536870912,"rowId":0,"currentTransaction":3,"row":null}
{"operation":2,"originalTransaction":1,"bucket":536936448,"rowId":0,"currentTransaction":3,"row":null}
________________________________________________________________________________________________________________________
acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000 [length: 705]
{"operation":2,"originalTransaction":2,"bucket":536870912,"rowId":0,"currentTransaction":3,"row":null}
{"operation":2,"originalTransaction":2,"bucket":536936448,"rowId":0,"currentTransaction":3,"row":null}
{noformat}
> SELECT statement fails after UPDATE with hive.optimize.sort.dynamic.partition
> optimization and vectorization on
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-20719
> URL: https://issues.apache.org/jira/browse/HIVE-20719
> Project: Hive
> Issue Type: Bug
> Components: Transactions, Vectorization
> Affects Versions: 4.0.0
> Reporter: Vineet Garg
> Assignee: Eugene Koifman
> Priority: Major
>
> *Reproducer*
> {code:sql}
> set hive.optimize.sort.dynamic.partition=true ;
> create table acid_uap(a int, b varchar(128)) partitioned by (ds string)
> clustered by (a) into 2 buckets stored as orc TBLPROPERTIES
> ('transactional'='true');
> insert into table acid_uap partition (ds='tomorrow') values (1, 'bah'),(2,
> 'yah') ;
> insert into table acid_uap partition (ds='today') values (1, 'bah'),(2,
> 'yah') ;
> select a,b,ds from acid_uap order by a,b;
> update acid_uap set b = 'fred';
> select a,b,ds from acid_uap order by a,b;
> {code}
> *Error*
> {code:java}
> Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1539123809352_0001_5_00,
> diagnostics=[Task failed, taskId=task_1539123809352_0001_5_00_000000,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1539123809352_0001_5_00_000000_0:java.lang.RuntimeException:
> java.lang.RuntimeException: java.io.IOException: java.io.IOException:
> Corrupted records with different bucket ids from the containing bucket file
> found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.io.IOException: Corrupted records with different bucket ids from the
> containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with
> different bucket ids from the containing bucket file found! Expected bucket
> id 0, however found DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> ... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids
> from the containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> ... 22 more
> ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
> attempt_1539123809352_0001_5_00_000000_1:java.lang.RuntimeException:
> java.lang.RuntimeException: java.io.IOException: java.io.IOException:
> Corrupted records with different bucket ids from the containing bucket file
> found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.io.IOException: Corrupted records with different bucket ids from the
> containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with
> different bucket ids from the containing bucket file found! Expected bucket
> id 0, however found DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> ... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids
> from the containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> ... 22 more
> ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) :
> attempt_1539123809352_0001_5_00_000000_2:java.lang.RuntimeException:
> java.lang.RuntimeException: java.io.IOException: java.io.IOException:
> Corrupted records with different bucket ids from the containing bucket file
> found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.io.IOException: Corrupted records with different bucket ids from the
> containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with
> different bucket ids from the containing bucket file found! Expected bucket
> id 0, however found DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> ... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids
> from the containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> ... 22 more
> ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) :
> attempt_1539123809352_0001_5_00_000000_3:java.lang.RuntimeException:
> java.lang.RuntimeException: java.io.IOException: java.io.IOException:
> Corrupted records with different bucket ids from the containing bucket file
> found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.io.IOException: Corrupted records with different bucket ids from the
> containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with
> different bucket ids from the containing bucket file found! Expected bucket
> id 0, however found DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> ... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids
> from the containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> ... 22 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:0, Vertex vertex_1539123809352_0001_5_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]
> Vertex killed, vertexName=Reducer 2, vertexId=vertex_1539123809352_0001_5_01,
> diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not
> succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex
> vertex_1539123809352_0001_5_01 [Reducer 2] killed/failed due
> to:OTHER_VERTEX_FAILURE]
> DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1,
> vertexId=vertex_1539123809352_0001_5_00, diagnostics=[Task failed,
> taskId=task_1539123809352_0001_5_00_000000, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1539123809352_0001_5_00_000000_0:java.lang.RuntimeException:
> java.lang.RuntimeException: java.io.IOException: java.io.IOException:
> Corrupted records with different bucket ids from the containing bucket file
> found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.io.IOException: Corrupted records with different bucket ids from the
> containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with
> different bucket ids from the containing bucket file found! Expected bucket
> id 0, however found DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> ... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids
> from the containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> ... 22 more
> ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
> attempt_1539123809352_0001_5_00_000000_1:java.lang.RuntimeException:
> java.lang.RuntimeException: java.io.IOException: java.io.IOException:
> Corrupted records with different bucket ids from the containing bucket file
> found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.io.IOException: Corrupted records with different bucket ids from the
> containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with
> different bucket ids from the containing bucket file found! Expected bucket
> id 0, however found DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> ... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids
> from the containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> ... 22 more
> ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) :
> attempt_1539123809352_0001_5_00_000000_2:java.lang.RuntimeException:
> java.lang.RuntimeException: java.io.IOException: java.io.IOException:
> Corrupted records with different bucket ids from the containing bucket file
> found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.io.IOException: Corrupted records with different bucket ids from the
> containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with
> different bucket ids from the containing bucket file found! Expected bucket
> id 0, however found DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> ... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids
> from the containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> ... 22 more
> ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) :
> attempt_1539123809352_0001_5_00_000000_3:java.lang.RuntimeException:
> java.lang.RuntimeException: java.io.IOException: java.io.IOException:
> Corrupted records with different bucket ids from the containing bucket file
> found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.io.IOException: Corrupted records with different bucket ids from the
> containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with
> different bucket ids from the containing bucket file found! Expected bucket
> id 0, however found DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> ... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids
> from the containing bucket file found! Expected bucket id 0, however found
> DeleteRecordKey(2,536936448(1.1.0),0). (OrcSplit
> [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000,
> start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false,
> hasBase=true,
> deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> ... 22 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:0, Vertex vertex_1539123809352_0001_5_00 [Map 1] killed/failed
> due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1539123809352_0001_5_01, diagnostics=[Vertex received Kill
> while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE,
> failedTasks:0 killedTasks:1, Vertex vertex_1539123809352_0001_5_01 [Reducer
> 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
