[
https://issues.apache.org/jira/browse/TEZ-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15411020#comment-15411020
]
Hitesh Shah edited comment on TEZ-3403 at 8/7/16 6:01 PM:
----------------------------------------------------------
There are 2 issues mentioned in the description. For the first one, please file
a Hive jira. For the second one, we can re-use this jira.
For both issues, please provide the following info:
- tez and hive versions
- which InputFormat you are using with the query in question
- yarn app logs
was (Author: hitesh):
There are 2 issues mentioned in the description. For the first one, please file
a Hive jira. For the second one, we can re-use this jira. Please attach yarn
application logs for the second issue.
For both issues, please provide the following info:
- tez and hive versions
- which InputFormat you are using with the query in question
> Empty partition issue with Hive on TEZ
> --------------------------------------
>
> Key: TEZ-3403
> URL: https://issues.apache.org/jira/browse/TEZ-3403
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Ashish Kumar
>
> Hi,
> I'm experiencing few failures with TEZ regarding Hive partitions. Even though
> there is no partition column used in the query still it is giving partition
> file path not found error.
> I'm trying to run below query with Hive on TEZ and getting some partition
> issue. The same query is working fine with MR engine. Used table is external
> one and having partitions on year and month columns. I've seen few times
> *Query:*
> *select count(crn) as bookings,
> month(to_date(from_utc_timestamp(pickup_date,'IST'))) as month from
> bookings_table and year=2016 group by
> month(to_date(from_utc_timestamp(pickup_date,'IST')));*
> *Error:*
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: While processing file
> s3n://<bucket>/warehouse/bookings_table/year=2016/month=1. null
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:78)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:292)
>
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
>
> ... 14 more
> Caused by: java.io.IOException: java.io.IOException: While processing file
> s3n://<bucket>/warehouse/bookings_table/year=2016/month=1. null
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
>
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
>
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:372)
>
> at
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79)
>
> at
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33)
>
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:118)
>
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:137)
>
> at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
>
> ... 16 more
> *Another error for other query:*
> DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:4
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1,
> vertexId=vertex_1470240409111_2339_1_06, diagnostics=[Vertex
> vertex_1470240409111_2339_1_06 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: app_sessions initializer failed,
> vertex=vertex_1470240409111_2339_1_06 [Map 1], java.io.FileNotFoundException:
> No such file or directory:
> s3n://<bucket>/warehouse/<table>/year=2015/month=02/day=14/hour=03
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1078)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.listStatus(S3AFileSystem.java:783)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1500)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1540)
> at org.apache.hadoop.fs.FileSystem$4.(FileSystem.java:1704)
> at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1703)
> at
> org.apache.hadoop.mapred.InputPathProcessor.perPathComputation(InputPathProcessor.java:235)
>
> at
> org.apache.hadoop.mapred.InputPathProcessor.access$000(InputPathProcessor.java:28)
>
> at
> org.apache.hadoop.mapred.InputPathProcessor$2.run(InputPathProcessor.java:338)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
